AI Can Build Your Test Framework. But Can You Still Maintain It?

Last month, I was reviewing a pull request from one of our automation engineers.

The code looked fantastic.

Clean methods.

Proper naming.

Reusable functions.

Good comments.

Exactly the kind of code every team likes to see.

I almost approved it.

Almost.

Something didn’t feel right.

So I asked him,

“Can you explain why you created this Retry Utility?”

He stared at the screen.

Then smiled awkwardly.

“Honestly… GitHub Copilot suggested it.”

I asked another question.

“Do you know how it works?”

Silence.

That silence lasted maybe five seconds.

But it told me everything.

After spending more than 15 years in automation testing, I’ve realized something that genuinely worries me.

AI is making automation engineers incredibly productive.

But sometimes…

it’s also making them incredibly dependent.

Not because AI is bad.

Because we’ve slowly stopped questioning what AI writes.

The Framework Was Built in Two Days

Ten years ago, building an automation framework took weeks.

You had to think about:

• Project structure
• Design patterns
• Driver management
• Configuration
• Reporting
• Logging
• Test data
• CI/CD

Today?

Open GitHub Copilot.

Ask ChatGPT.

Within two hours you have:

• Page Objects
• Driver Factory
• Base Classes
• Report Manager
• Utilities
• Jenkins pipeline
• GitHub Actions

It feels magical.

But here’s my question.

If the framework breaks six months later…

Can you fix it without AI?

That’s the real interview question.

Case Study 1 — The Retry That Hid Every Bug

A team started using GitHub Copilot for automation development.

One flaky test kept failing.

Copilot suggested adding a Retry mechanism.

Simple.

Elegant.

Problem solved.

Or so they thought.

Three months later…

A production bug escaped.

The Login API was intermittently returning HTTP 500.

Automation never reported it.

Why?

Because every failure was automatically retried until it passed.

The framework became quieter.

Not better.

The retry utility didn’t fix instability.

It simply hid it.

AI solved the symptom.

Nobody investigated the disease.

AI Writes Based on Patterns. Your Application Isn’t a Pattern.

This is something many testers forget.

Copilot has never worked on your application.

It doesn’t know:

• Your business rules
• Your architecture
• Your customers
• Your production incidents
• Your technical debt

It predicts the most likely code.

Sometimes…

the most likely code is completely wrong.

Case Study 2 — The Perfect Page Object

A junior automation engineer generated an entire Page Object using AI.

Beautiful code.

Every locator was stored properly.

Methods were reusable.

Everything looked professional.

Until one developer changed a button ID.

Suddenly…

47 test cases failed.

The junior engineer spent two days fixing them.

I asked,

“Why didn’t you create dynamic locators?”

His answer was honest.

“Copilot generated static ones, so I assumed that’s the best approach.”

That’s the danger.

Not bad code.

Blind trust.

Automation Engineers Are Slowly Becoming Operators

Years ago…

Building automation meant understanding architecture.

Today I sometimes see this workflow.

Requirement

↓

Ask AI

↓

Copy Code

↓

Run Tests

↓

Merge PR

↓

Repeat

Notice what’s missing?

Thinking.

Nobody asks,

• Why is this utility needed?
• Should this class exist?
• Is this reusable?
• Will this scale?
• What happens after two years?

Frameworks don’t become difficult overnight.

They slowly become unmaintainable because nobody questions their design.

Case Study 3 — The Utility Class That Became a Monster

One project had a Utility class with nearly 3,800 lines of code.

Every new feature…

Copilot suggested,

“Add another helper method.”

Eventually it contained:

• Screenshot methods
• Excel readers
• API utilities
• Database queries
• JSON parsers
• Date generators
• Browser utilities
• File handling

Everything.

Developers hated touching it.

Automation engineers feared modifying it.

One small change broke twenty unrelated tests.

AI didn’t create the mess.

Years of unquestioned AI suggestions did.

The Biggest Bug Copilot Introduces Isn’t in Code

It’s in your mindset.

When AI writes everything…

You stop asking why.

When AI fixes failures…

You stop debugging.

When AI writes SQL…

You stop learning databases.

When AI builds frameworks…

You stop understanding architecture.

Slowly…

Your confidence increases.

Your knowledge decreases.

That’s dangerous.

Because one day production fails.

AI won’t join your incident bridge call.

You will.

What I Use AI For Every Single Day

Don’t misunderstand me.

I use AI constantly.

GitHub Copilot saves me hours every week.

I use it to generate:

• Selenium code
• Playwright scripts
• API requests
• SQL queries
• Python utilities
• Framework templates
• Documentation

AI is an incredible productivity partner.

But never an unquestioned architect.

Every AI-generated line must answer one question.

“Would I have written this if AI didn’t exist?”

If the answer is No…

I stop.

I understand it first.

Only then do I keep it.

My Personal Rule

Whenever Copilot generates code, I force myself to answer these questions before pressing Commit.

• Why is this code needed?
• Can I explain it to a junior engineer?
• What problem does it solve?
• What are its side effects?
• How will this behave after 10,000 test executions?
• What happens if another engineer maintains it?

If I can’t answer those questions…

I don’t merge it.

No matter how beautiful the code looks.

The Future Belongs to Engineers, Not Prompt Writers

Five years from now…

Everyone will have AI.

Everyone will have Copilot.

Everyone will generate automation in minutes.

That won’t make you valuable.

Your value will come from something AI cannot easily replicate.

Good engineering judgment.

Understanding architecture.

Predicting failures.

Questioning assumptions.

Thinking about tomorrow instead of today’s commit.

Because frameworks aren’t judged by how quickly they’re built.

They’re judged by how easily they’re maintained five years later.

Final Thoughts

The biggest misconception today is that AI is replacing automation engineers.

I don’t believe that.

AI is replacing repetitive coding.

That’s a good thing.

But if we allow AI to replace our curiosity…

our design thinking…

our debugging skills…

our understanding of architecture…

then we are quietly replacing ourselves.

Remember this the next time GitHub Copilot writes 200 lines of perfect code for you.

Don’t ask,

“Does this compile?”

Ask,

“Will I still understand this six months from now?”

Because AI can build your automation framework in a day.

Only an engineer can keep it alive for years.

A Note to Readers

The conversation at the beginning of this article is an imaginary scenario created to illustrate a situation that many QA engineers may experience in today’s AI-driven world. It is not based on any specific individual, colleague, or team member.

The examples and case studies throughout this blog are inspired by common industry patterns, real-world testing challenges, production incidents, and lessons learned over many years in software testing.

The purpose of this article is not to criticize AI or the engineers using it. In fact, I use AI every day and believe it is one of the greatest productivity tools available to testers.

The real message is simple:

Use AI to write code. Never let it replace your thinking.

If this blog makes you pause for a moment before accepting AI-generated code, ask one more question during testing, or spend a few extra minutes understanding your automation framework, then it has achieved its purpose.