AI governance for small and mid-size businesses: a practical framework

Most AI governance content is written for Fortune 500 legal teams or academic researchers. Neither audience is a $20M manufacturing company that started using ChatGPT for proposals, added an AI hiring screener six months later, and is now wondering if any of this creates legal exposure.

It does. And the window to get ahead of it is closing.

The EU AI Act reached full enforcement on August 2, 2026. US executive orders on AI are tightening procurement and liability expectations. Enterprise buyers are adding AI policy questionnaires to vendor due diligence. If your business uses AI tools and you have not documented how, you are already behind.

This post is a practical starting point for businesses doing $5M–$100M in revenue. No enterprise software purchase required.

Why AI governance matters now for mid-market companies

The pressure is coming from three directions at once.

Regulation. The EU AI Act is not a future concern. Prohibited practices have been enforceable since February 2025. Full enforcement including high-risk AI system rules kicked in August 2, 2026. If your business sells to EU customers or your AI systems are used by EU residents, you are in scope regardless of where your company is based. The same extraterritorial logic that made GDPR a US business problem makes the AI Act one too.

Fines for high-risk violations reach up to EUR 35 million or 7% of global annual turnover. For a company with EUR 25M in revenue, the maximum penalty after SME reductions is around EUR 375,000. That is not an abstract enterprise risk.

In the US, regulated industries face their own pressure. Healthcare organizations using AI that touches protected health information need a Business Associate Agreement with every AI vendor. Most consumer AI tools do not offer this. Financial services firms face SEC and FINRA guidance on AI in investment advice and explainability requirements that are already active.

Liability exposure. If an AI agent makes a discriminatory hiring decision, or an AI credit-scoring tool denies someone a loan on a faulty basis, the question of who is responsible points back to the deployer. That is your business. Gartner predicts AI regulatory violations will result in a 30% increase in legal disputes for tech companies by 2028. The liability is not theoretical.

Customer and buyer trust. Enterprise procurement teams are adding AI policy questionnaires to vendor due diligence checklists. One common pattern: enterprise buyers discovered that vendors' employees were pasting confidential client data into AI tools that retained it for model training. That kind of incident ends vendor relationships. If your business sells to other businesses, expect to be asked about your AI governance policy before contracts renew.

58% of small businesses now use generative AI, up from 40% in 2024. Adoption is accelerating faster than governance is following. That gap is where the liability lives.

What AI governance actually is

Enterprise vendors will tell you governance requires a six-figure platform purchase with model monitoring dashboards, bias detection APIs, and a dedicated risk team. That is governance at scale, and it is not where most SMBs need to start.

At its core, AI governance is a set of policies and processes for:

• Which AI tools your business is allowed to use

• What data can go into each AI system

• Who has authority to approve new AI tools

• How AI-generated outputs are reviewed before they affect customers or decisions

• What happens when an AI system makes a mistake

None of these require software. They require decisions and documentation.

The three components to get right are:

Data governance. Which data can touch which AI tools. The risk is concrete: putting client data into a public AI model that uses it for training, putting PHI into a tool with no BAA, putting financial records into a vendor with unclear data retention policies.

Model governance. Which AI models and tools are approved, why they were selected, and what the process is for evaluating new ones. This is a procurement policy, not a technology problem.

Process governance. Where in your workflow AI outputs require human review before they affect a customer or a business decision. An AI draft of a contract needs a lawyer's eyes. An AI hiring screen needs a recruiter's review before a candidate is rejected.

The minimum viable version of all three of these fits in a shared document and three policy memos. Start there.

The EU AI Act: what SMBs actually need to know

The EU AI Act organizes AI systems into four risk tiers. Understanding which tier applies to your tools tells you what you actually have to do.

Unacceptable risk. Prohibited outright. AI systems that manipulate people through subliminal techniques, social scoring systems, real-time biometric surveillance in public spaces (with narrow exceptions). No legitimate business AI tool falls here.

High risk. These face the full weight of EU AI Act obligations: technical documentation, risk management systems, human oversight requirements, detailed logging, and conformity assessments. The categories that affect SMBs are:

• AI used in recruitment: CV screening, candidate ranking, interview evaluation, performance monitoring that informs promotion or dismissal

• Credit scoring and creditworthiness assessment

• AI that affects access to essential services

• Biometric identification systems

If you have deployed an AI hiring tool, even a third-party SaaS product, you are a "deployer" under the EU AI Act and subject to high-risk obligations. The Act regulates functional roles in the AI value chain, not company size.

Limited risk. Transparency obligations apply. If you use a chatbot, you must tell users they are interacting with AI. If AI generates content that could be mistaken as human-created, you must label it. This is the tier where most customer-facing AI tools live.

Minimal risk. Spam filters, AI-powered search, recommendation engines in most contexts. Effectively unregulated.

In practice, the vast majority of business AI tools fall into limited or minimal risk. The governance work for those tiers is not heavy. The meaningful exposure for most SMBs is: do you use AI in hiring, credit decisions, or anything that affects access to services? If yes, treat those systems as high-risk and build accordingly.

A practical governance framework for SMBs: the 5-layer model

This framework does not require a platform purchase. It requires four to six weeks of internal work the first time, and a few hours per month to maintain.

Layer 1: AI inventory

Document every AI tool your business uses. Start broad: standalone AI tools (ChatGPT, Claude, Gemini), AI features embedded in SaaS products (AI in your CRM, ATS, customer service platform, email marketing tool), and any custom-built AI features in your product.

For each tool, record:

• Tool name and vendor

• What it is used for

• Which team uses it

• What data goes into it

• Whether a vendor contract and data processing agreement exist

More than half of organizations do not know what AI systems they are currently using. The inventory is the prerequisite for everything else. You cannot classify risk or check vendor compliance for tools you have not listed.

Layer 2: Data classification

Define which data can enter which AI systems. Three practical categories:

Public data only. Information already public or fully anonymized. Can go into any tool, including consumer-tier AI products.

Internal business data. Internal documents, company financials, non-customer business information. Requires a vendor data processing agreement before it goes into any third-party AI tool. Should not go into consumer-tier tools at all.

Sensitive or regulated data. PHI, PII, payment card data, client confidential information. Requires a signed BAA (for health data), DPA, or equivalent legal agreement. Should only enter AI systems explicitly designed and certified for that data class.

The most common violation we see: employees pasting client data or internal financial information into public-tier AI tools because there was no policy telling them not to. A one-page data classification policy, shared with every employee who uses AI tools, eliminates the majority of unintentional data exposure.

Layer 3: Approved model and tool list

Establish a procurement policy for AI tools. Before any new AI tool is added to the business:

1. Identify the data it will access
2. Review the vendor's data processing and retention terms
3. Confirm the vendor offers the legal agreements required for that data class (BAA, DPA, etc.)
4. Confirm the tool does not use your data to train shared models (or that you have opted out)
5. Get sign-off from whoever owns your legal and data decisions

This does not need to be a formal committee. It needs to be a defined process so that "we just started using it" is not how new AI tools enter the business.

Layer 4: Human review checkpoints

Define where in your workflows AI output requires human review before action is taken. Some examples:

• AI-drafted customer-facing communications: reviewed by a human before sending

• AI hiring screen: a recruiter reviews before any candidate is rejected

• AI-generated contract language: reviewed by legal before use

• AI financial analysis: reviewed by finance before informing a decision

• AI support ticket resolution: reviewed when the resolution affects a refund, cancellation, or policy exception

The rule of thumb: any AI output that, if wrong, would harm a customer, expose the business to liability, or result in a significant incorrect decision needs a human checkpoint.

Layer 5: Incident response

Define what happens when an AI system makes a mistake that affects a customer or a business decision. At minimum:

• How is the incident reported internally

• Who reviews it

• How are affected customers notified

• Is there a regulatory notification obligation (HIPAA breaches, EU AI Act incident reporting for high-risk systems)

• What is the process for deciding whether to continue using the tool

Most businesses have no answer to "what do we do if the AI hiring tool rejected a qualified candidate because of a data error?" Having the answer written down before an incident matters because the pressure in the moment is to minimize, not document.

What regulated industries need beyond the baseline

The five-layer framework is the floor. Regulated industries need additional specifics.

Healthcare

HIPAA requires a signed Business Associate Agreement with any vendor that processes PHI. This is not optional and it is not covered by standard vendor terms of service.

Most general-purpose AI tools (ChatGPT, Claude, Gemini in standard tiers) do not offer BAAs and are not HIPAA-eligible for use with patient data. Enterprise tiers of some models do offer BAAs, but verify before use. Healthcare-specific AI platforms are built for this context and typically provide the contractual and technical controls HIPAA requires.

Beyond the BAA: your AI governance policy should explicitly prohibit the use of PHI in any AI tool that has not signed a BAA. Include this in onboarding and in regular staff training.

The proposed January 2025 updates to the HIPAA Security Rule would further raise the bar for any system handling electronic PHI. The direction of travel is toward more documentation and stronger access controls, not less.

Financial services

SEC and FINRA have issued guidance on AI in investment advice contexts. The key requirement is explainability: if AI influences a recommendation to a client, you need to be able to explain how that recommendation was generated and demonstrate that it is not biased.

For credit decisions, the EU AI Act high-risk classification applies. You also have obligations under US fair lending law (ECOA, Fair Housing Act) not to use AI in ways that produce discriminatory outcomes, regardless of intent.

Insurance

State insurance commissioners have begun issuing guidance on AI use in underwriting. Several states require that AI used in underwriting decisions be explainable and not produce discriminatory outcomes. The National Association of Insurance Commissioners (NAIC) model bulletin on AI provides a baseline, but state-by-state variation is significant.

If you use AI in underwriting, claims processing, or pricing decisions, treat it as high-risk and build the documentation and review processes accordingly.

Where to start: a 30-day checklist

You do not need to implement the full framework in week one. This is a realistic starting sequence.

The full framework is operational at the end of 30 days. It will not be perfect. It will be better than having nothing, and better than the exposure you currently have.

Only 1.5% of surveyed organizations believe they have adequate governance headcount. The point is not to hire a governance team. The point is to make governance someone's responsibility, even if that someone has other responsibilities too.

The cost question

AI governance platforms are a real product category. Gartner projects the market will reach $492 million in 2026 and surpass $1 billion by 2030. Those platforms make sense for organizations with dozens of AI systems, regulatory audit requirements, and dedicated compliance teams.

For most SMBs, the minimum viable governance framework costs almost nothing out of pocket. The investment is staff time: a few days to build the inventory and write policies, a few hours per month to maintain them.

If you operate in a regulated industry or have significant legal complexity, budget $5,000–$20,000 for outside counsel to review your vendor contracts and policies. That is not a platform purchase. It is legal review, which you would need anyway.

Add tooling after your processes are defined. Buying a governance platform before you have policies is buying structure for a building you have not designed.

FAQs

Does the EU AI Act apply to US businesses? Yes, if you offer products or services to people in the EU, or if AI systems you deploy are used by EU residents, the EU AI Act applies regardless of where your company is based. The same extraterritorial logic applies to GDPR. If you have EU customers or EU-based users, you are in scope.

What is the first step to AI governance for a small business? Build an AI inventory. List every AI tool your business uses. You cannot classify risk, set data policies, or check vendor compliance until you know what you are running.

Which AI tools are high-risk under the EU AI Act? Tools used for hiring decisions (CV screening, interview scoring, performance monitoring), credit scoring, access to essential services, and biometric identification. Most SaaS productivity tools, writing assistants, and customer service chatbots are not high-risk.

Do I need a BAA for using AI tools with customer data in healthcare? Yes. Any vendor processing PHI on your behalf must sign a BAA. Most consumer AI tools do not offer this. Never put patient data into a tool that has not signed a BAA with you.

How much does implementing AI governance cost for a mid-market business? The minimum viable framework costs very little out of pocket. The main investment is staff time. If you have legal complexity (HIPAA, financial services), budget $5,000–$20,000 for outside counsel to review your policies and vendor contracts. You do not need a platform subscription to start.

If you're building AI features into your product and need guidance on governance requirements, talk to our team.