---
title: "On-Device vs Cloud AI: Where Small Businesses Should Keep Customer Data"
url: https://josephleporati.com/blog/on-device-vs-cloud-ai-privacy-small-business
author: Joseph Leporati (https://josephleporati.com/)
location: Monroe, Louisiana
published: 2026-11-02
updated: 2026-11-02
places: ["Monroe","West Monroe","Ruston"]
---

# On-Device vs Cloud AI: Where Small Businesses Should Keep Customer Data

Keep customer records, scheduling data and stored chat transcripts in systems you control. Run identifier redaction and intent routing on a local model, and send only redacted, minimal context to a cloud LLM for the open-ended conversation. Do that under a contract that bars training on your data and sets retention limits. Running the model locally does not move liability: under FTC guidance and Louisiana breach law, protecting your customers' data is still your job.

Most small businesses treat this as a single choice between local and cloud. In practice you make it once for each data flow. The chat transcript, the customer record the assistant looks up, the calendar it writes to, and the logs nobody remembers each carry different risk, and each has a different best place to live.

This article covers how each design moves data, where on-device models fall short, how to split the work, and how that split changes your vendor contracts and your exposure under Louisiana and federal rules.

## What actually leaves the building in each design?

There are three architectures you will see in practice.

**Fully local (on-device or on-premises).** The model weights run on a machine you control. Prompts and outputs never go to a model provider. You own the hardware, the patching, the logs and the backups.

**Cloud LLM, direct.** Your app or a hosted chatbot sends the whole conversation, plus whatever customer context it pulled, to a provider's API. With a hosted chatbot there are often two vendors: the chatbot company and the model provider behind it. Each has its own retention policy and subprocessors. The path usually looks like this: website widget, chatbot vendor, model API, then vendor logs and analytics.

**Hybrid with a local gateway.** A small local model or plain deterministic code sits between your systems and the cloud. It strips identifiers, picks the task, fetches only the fields that task needs, and sends a redacted prompt to the cloud model. Real values are put back locally before the customer sees the reply.

For most small businesses I recommend the hybrid. You get cloud-quality conversation without sending your customer table to anyone.

## Is an on-device model actually more private?

It is more private only at the inference step. Everything around the model can still leak, and these are the failure modes I see most often:

- **Telemetry.** Local model runtimes and the desktop apps that wrap them can send crash reports or usage analytics. Check the settings and watch outbound traffic.
- **Tool calls.** If the local assistant can search the web, send email or call a booking API, data leaves through those paths.
- **Logs and backups.** Transcripts written to disk in plaintext, then synced to a consumer cloud backup account, end up in the cloud anyway, with weaker controls than a business API contract.
- **Physical security.** A workstation under a desk in a West Monroe office is a weaker perimeter than a well-run data center. A stolen laptop with an unencrypted transcript folder is a breach.
- **Quality.** Smaller local models are more likely to make things up in open conversation. An assistant that confirms an appointment slot that doesn't exist causes real harm, even though no data leaked.

Wherever the model runs, Louisiana's [Database Security Breach Notification Law (R.S. 51:3074)](https://legis.la.gov/legis/LawPrint.aspx?d=322030) requires any business that owns or licenses computerized personal data to maintain reasonable security procedures appropriate to that information. "Local" does not count as a security program by itself.

## Which parts of the assistant can run locally?

Split the work by task and by data. This is the split I start from:

| Task or data | Sensitivity | Where it should run | Why |
|---|---|---|---|
| Detecting and redacting names, phones, emails, addresses | High | Local (patterns plus a small model) | Narrow task that small models handle well, and it has to happen before anything leaves |
| Intent classification and routing | Low after redaction | Local | Small label set and cheap inference |
| Customer record lookup | High | Your database, queried by code | The model should never have bulk read access |
| Scheduling availability and booking writes | Medium | Your system, through a validated tool call | The model proposes a booking and your code commits it |
| Open-ended reply drafting | Depends on prompt contents | Cloud LLM, on redacted input | Clear quality gap at small-business budgets |
| Stored chat transcripts | High | Your storage, encrypted, with a retention limit | Usually the largest pile of personal data the assistant creates |
| Summaries of sensitive documents (intake forms, claims) | High | Local, or cloud only under a strict contract | Decide per document type |

The core of the hybrid fits in a few lines:

```python
def handle_message(session, text):
    # 1. Local: replace identifiers with placeholder tokens
    redacted, vault = local_redact(text)        # "Call me at 318..." -> "Call me at <PHONE_1>"

    # 2. Local: route the request and fetch only what this task needs
    intent = local_classify(redacted)           # e.g. "reschedule"
    slots = scheduling.open_slots(
        customer_id=session.customer_id,        # from the authenticated session, never from model output
        service=intent.service,
    )

    # 3. Cloud: reasoning on tokens and minimal context, no raw records
    reply = cloud_llm.complete(system=POLICY, user=redacted, context=slots.public_view())

    # 4. Local: put real values back before the customer sees the reply
    return rehydrate(reply, vault)
```

Two details matter more than the model choice. First, take the customer ID from the authenticated session, never from text the model produced, so a prompt injection can't fetch someone else's record. Second, booking writes go through code that checks the slot exists and belongs to that customer. The model's output is a suggestion. Your code makes the decision. I use the same rule in [AI assistants that only answer from cited records](/blog/ai-that-cites-sources-local-government-louisiana): the model never gets to be the source of truth.

If the assistant runs on phones or tablets out in the field with poor connectivity, local redaction and queued sync work together well. The merge problems are covered in [offline-first field apps](/blog/offline-first-mobile-app-conflict-resolution).

## What the choice means for vendor contracts and liability

### You stay responsible for the vendor's behavior

The FTC's position, [as summarized by Centrex IT](https://centrexit.com/blog/ftc-cppa-ai-enforcement-smb-compliance-exposure/), is that small businesses using third-party AI tools remain responsible for outcomes whether or not they understand the underlying technology. Choosing a reputable vendor does not hand off that responsibility. What the vendor promises about training, retention and access only protects you if it is written into the contract and you can check it.

The FTC has also warned that it may be unfair or deceptive to adopt more permissive data practices, such as sharing data or using it for AI training, and tell consumers only through a [quiet, retroactive change to the privacy policy](https://www.consumerfinancemonitor.com/2024/02/28/ftc-warns-quietly-changing-privacy-policies-may-be-an-unfair-or-deceptive-practice/). If adding an assistant changes where customer data goes, update your policy before launch, not afterward.

### Louisiana breach timelines

If personal data leaks, Louisiana requires notice to affected residents without unreasonable delay and [no later than 60 days after discovery](https://www.insureon.com/small-business-insurance/cyber-liability/data-breach-laws/louisiana). The Attorney General must [receive notice within 10 days](https://www.leakeandersson.com/louisiana-strengthens-its-database-security-and-privacy-laws/) of the notices going to residents. If your vendor finds a breach and takes three weeks to tell you, a third of your window is already gone. That is why the contract's notification clause matters.

### The Louisiana Data Privacy Act

Governor Landry signed the [Louisiana Data Privacy Act on May 29, 2026](https://www.dataguidance.com/news/louisiana-data-privacy-act-signed-governor), effective January 1, 2027. It applies to businesses with [$25 million or more in annual revenue, personal data of 75,000 or more consumers a year, or 50% or more of revenue from selling personal data](https://www.didomi.io/blog/louisiana-data-privacy-act-ldpa). Most small businesses in Monroe and West Monroe will fall below those thresholds. It exempts GLBA financial institutions and HIPAA-covered entities, and it has [no private right of action](https://www.wilmerhale.com/en/insights/blogs/wilmerhale-privacy-and-cybersecurity-law/20260622-louisiana-enacts-nations-twenty-second-state-comprehensive-privacy-law). Covered controllers must answer consumer requests [within 45 days, with one 45-day extension](https://www.hunton.com/privacy-and-cybersecurity-law-blog/louisiana-enacts-comprehensive-consumer-privacy-law). A 30-day cure period runs only through July 31, 2027.

Even below the thresholds, the hybrid design pays off. When the customer data lives in one database you control, an access or deletion request is a query you run yourself. Without that, you're filing a ticket with two vendors.

### Contract checklist

Centrex IT recommends reading AI vendor contracts specifically for the [data processing addendum, the indemnification clause and training-data language](https://centrexit.com/blog/ftc-cppa-ai-enforcement-smb-compliance-exposure/). I'd add a few items. Before you sign, confirm:

1. **Training:** your inputs and outputs are not used for training, and that cannot change without notice and your consent.
2. **Retention:** how long prompts, outputs and logs are kept, and deletion on termination, confirmed in writing.
3. **Subprocessors:** which model provider sits underneath, and whether you get notice when it changes.
4. **Breach notice to you:** a specific window short enough for you to meet the 60-day and Attorney General deadlines.
5. **Indemnification:** who pays for notification and remediation when the vendor causes the breach.
6. **Access:** which vendor staff can read transcripts, and under what conditions.

When a fully local design is chosen mainly to avoid this list, that is usually the wrong reason. You'll still have contracts with your hosting, backup and remote-support providers.

## Is running AI locally cheaper than a cloud subscription?

At small-business volume it usually isn't, in my experience. Cloud API fees scale with tokens, so the bill depends on three things: how many conversations you handle, how long each prompt is (system instructions and retrieved context count, not just the customer's message), and which model tier you pick. The honest way to estimate is to take a week of real transcript volume, measure average prompt and reply length, and multiply by the provider's published per-token price. The hybrid design also lowers the bill, because redaction and routing trim what gets sent.

A local deployment costs more than the hardware. These are the drivers:

| Cost driver | Local | Cloud API |
|---|---|---|
| Hardware | GPU-capable machine sized for the model and peak concurrent users | None |
| Operations | OS and runtime patching, disk encryption, backups, monitoring, on-call | Mostly the provider's job |
| Model upkeep | Testing and rolling out newer model versions yourself | Version changes announced by the provider, still worth regression testing |
| Engineering | Redaction gateway and tool-call validation | Same gateway and validation (the hybrid needs them too) |
| Usage | Fixed, regardless of volume | Grows with conversations and prompt length |

Choose local for control and data minimization, not to save money. If you want local talent for GPU inference work, Louisiana Tech in Ruston is [preparing to open a 15-station NVIDIA AI Laboratory](https://www.latech.edu/news/listing.php).

## Where to start

1. **Inventory every AI tool in use,** including support widgets and AI features switched on inside software you already pay for. [Centrex IT's checklist](https://centrexit.com/blog/ftc-cppa-ai-enforcement-smb-compliance-exposure/) starts here too.
2. **Draw the data flow for each one:** what enters, which vendors touch it, where it's logged, how long it's kept.
3. **Mark the fields that never leave** (full names, phones, addresses, account numbers) and put redaction in front of every cloud call.
4. **Set a retention limit on transcripts** and enforce it with a scheduled job.
5. **Pull the contracts** and check them against the list above.
6. **Write the breach runbook** with the 60-day and 10-day clocks in it.

If you want structured help with the vendor and architecture side, that's the kind of work a [fractional CTO](/blog/fractional-cto-monroe-louisiana) takes on. The Monroe Chamber's [AI Lunch & Learn series](https://www.knoe.com/2026/07/17/monroe-chamber-commerce-technology-committee-host-ai-lunch-learn-event/) also runs through November 19, 2026.

If you want a second opinion on your assistant's data flow or vendor contract, email hello@josephleporati.com.

## Common questions

### If I run the AI model locally, do I still need vendor contracts?

Usually, yes. A local model removes the model provider from the data path, but your hosting, backup, remote-support and scheduling vendors still handle customer data. Louisiana's breach law requires reasonable security for computerized personal data no matter who touches it, so each of those vendors needs clear terms on retention, access and how fast they will tell you about a breach.

### How long should an AI assistant keep chat transcripts?

Only as long as you have a specific use for them, such as resolving a dispute or reviewing the assistant's quality. Pick a fixed window, store transcripts encrypted in a system you control, and delete them with a scheduled job instead of by hand. Transcripts are often the largest store of personal data an assistant creates, so a short retention period reduces breach exposure more than almost any other single control.

### Can I add an AI chatbot and just update my privacy policy afterward?

That is risky. The FTC has warned that adopting more permissive data practices, such as sharing data with third parties or using it for AI training, and disclosing them only through a quiet, retroactive change to your terms or privacy policy may be unfair or deceptive. Update the policy to describe the new data flow before the assistant goes live.

### What should I do if my AI vendor tells me about a breach?

Start your clock right away. Louisiana requires notice to affected residents no later than 60 days after discovery, and the Attorney General must receive notice within 10 days of the resident notices going out. Get the vendor's scope details in writing, confirm which customers were affected, contact your insurer and counsel, and follow your written breach runbook.

## Sources

1. [Louisiana R.S. 51:3074, Database Security Breach Notification Law](https://legis.la.gov/legis/LawPrint.aspx?d=322030)
2. [Insureon: Louisiana data breach laws](https://www.insureon.com/small-business-insurance/cyber-liability/data-breach-laws/louisiana)
3. [Leake & Andersson: Louisiana strengthens its database security and privacy laws](https://www.leakeandersson.com/louisiana-strengthens-its-database-security-and-privacy-laws/)
4. [DataGuidance: Louisiana Data Privacy Act signed by governor](https://www.dataguidance.com/news/louisiana-data-privacy-act-signed-governor)
5. [Didomi: Louisiana Data Privacy Act (LDPA)](https://www.didomi.io/blog/louisiana-data-privacy-act-ldpa)
6. [WilmerHale: Louisiana enacts comprehensive privacy law](https://www.wilmerhale.com/en/insights/blogs/wilmerhale-privacy-and-cybersecurity-law/20260622-louisiana-enacts-nations-twenty-second-state-comprehensive-privacy-law)
7. [Hunton Andrews Kurth: Louisiana enacts comprehensive consumer privacy law](https://www.hunton.com/privacy-and-cybersecurity-law-blog/louisiana-enacts-comprehensive-consumer-privacy-law)
8. [Centrex IT: FTC and CPPA AI enforcement and SMB compliance exposure](https://centrexit.com/blog/ftc-cppa-ai-enforcement-smb-compliance-exposure/)
9. [Consumer Finance Monitor: FTC warns quietly changing privacy policies may be unfair or deceptive](https://www.consumerfinancemonitor.com/2024/02/28/ftc-warns-quietly-changing-privacy-policies-may-be-an-unfair-or-deceptive-practice/)
10. [Louisiana Tech University news](https://www.latech.edu/news/listing.php)
11. [KNOE: Monroe Chamber Technology Committee AI Lunch & Learn](https://www.knoe.com/2026/07/17/monroe-chamber-commerce-technology-committee-host-ai-lunch-learn-event/)
