The Questions to Ask Before Hiring a Custom Software Agency

Hiring a custom software agency is a multi-year decision disguised as a project. The system an agency builds will keep running long after the invoice is paid, and the questions that matter most are usually the ones a sales conversation skips over. This is a working checklist — organized by the categories that tend to cause real problems later — for evaluating any agency you’re considering.

This checklist covers the full lifecycle of custom software, including architecture, change control, launch, maintenance, and an eventual vendor transition. For an engagement centered on AI-assisted automation, use the companion AI automation agency scorecard to examine model behavior, data access, human review, and AI-vendor risk in more depth.

Fit

Have they asked about your actual process before proposing a solution, or did a specific technology come up before they understood the problem?
Do they have experience with businesses of your size and complexity, or mostly with a different scale of client?
Would they tell you if custom software isn’t the right answer — or does every conversation end in a build recommendation?
Who would actually be doing the work: the people in the sales conversation, or a different team you haven’t met yet?

Process

What does their discovery phase actually produce — a written scope and plan, or just a verbal understanding?
How do they handle scope changes discovered mid-project? Is there a defined process, or does everything get renegotiated informally?
How often will you hear from them during the build, and through what channel?
What happens if your internal priorities shift halfway through the engagement?

Data

Where will your business data live during and after the project — your infrastructure, theirs, or a third-party vendor’s?
Who owns the data the system generates or processes once the engagement ends?
What happens to your data if you switch providers or end the relationship?
Will they document exactly what data flows where, or does that knowledge stay undocumented in someone’s head?

Security

What access controls will exist around the system — who can see, edit, or export sensitive data?
Will they sign a data processing agreement, and if you handle regulated data, a business associate agreement or equivalent?
How are credentials, API keys, and secrets managed — and are they ever visible in code, chat logs, or shared documents?
Do they have a documented process for handling a security incident, or is that being figured out for the first time if it happens?

Architecture

Will the system be built on technology and patterns your own team (or a future agency) could reasonably take over, or on something proprietary to this one vendor?
How does it integrate with the systems you already depend on — through documented APIs, or fragile workarounds?
What happens to the architecture as your usage grows — has anyone thought past the initial version?
Is the codebase and infrastructure something you can inspect, or is it a black box you’re expected to trust?

Ownership and Handoff

Who owns the code, the infrastructure accounts, and the domain or hosting credentials when the project is done?
Will you receive documentation sufficient for someone else to maintain the system if this agency is no longer involved?
Is there a formal handoff step, or does the relationship continue by default with no clear endpoint?
If you wanted to bring the work in-house or move to another vendor, what would that actually require?

Testing

How is the system tested before it reaches real users — and does that include the edge cases specific to your business?
Who reviews outputs before they reach a customer, especially for anything AI-assisted or automated?
What’s the process for testing changes after launch, before they go live?
Will you get to test the system yourself with real scenarios before sign-off, or only see a demo?

Change Control

How are change requests submitted, estimated, and approved once the initial scope is built?
Is there a record of what changed, when, and why — or does the system’s behavior drift over time without a trail?
Who has the authority to approve a change that affects cost or timeline?
What happens if a requested change conflicts with something already built?

Launch

What does “done” actually mean for this project — a specific, written definition, or a general sense that it works?
Is there a rollback plan if something goes wrong immediately after launch?
Who is responsible for monitoring the system in the first days and weeks after it goes live?
Will your team be trained before launch, or expected to learn by using it?

Maintenance

What ongoing support is available after launch, and what does it actually cover — bug fixes, monitoring, feature requests, all of the above?
What’s the expected response time if something breaks in production?
Who is responsible for keeping dependencies, integrations, and security patches current over time?
Is maintenance optional, or does the system realistically require it to keep working as the tools around it change?

Exit Planning

If you ended the relationship tomorrow, could you keep the system running without this agency?
Is there a transition plan on file, or would one need to be built from scratch under time pressure?
What contractual notice or transition period applies if either side wants to end the engagement?
Has anyone actually asked this question before you signed anything — or is it being asked here for the first time?

Why This List Matters More Than the Pitch

A capable agency should be able to answer most of these questions clearly and specifically, without hedging into vague reassurance. Vague answers on ownership, data, and exit questions are a red flag. Good agencies usually welcome this list.

For a look at how we specifically answer the process, ownership, and support questions above, see how D3FY scopes, builds, and supports custom software. And if data handling is the part you’re least sure how to evaluate, our piece on AI and data security for small businesses goes deeper on that category specifically.

If you’d rather work through this list with someone before signing anything, an AI readiness audit is a low-commitment way to get a second opinion, or you can talk directly with our custom software and AI consulting team about your specific project.