Why QAmeleon exists
Most small teams don't have a QA person. The founder tests what they can, the developer tests what they wrote, and the rest goes out and hopes for the best.
That isn't laziness. It's that nobody has the time, and a lot of founders don't know what they don't know. They aren't sure what to check, so they never feel sure about what they ship.
QAmeleon is a QA engineer for those teams. It uses your app the way your users do, checks every ticket, and tells you in plain words what to fix.
Who is building it
I'm Eamonn, and I'm building QAmeleon part-time, from Co. Kildare in Ireland.
I spent over 20 years running kitchens and catering teams before moving into software QA after COVID. Since then I've tested products in health testing, nutrition data, energy and mobile apps, including work where a missed bug has real consequences for real people.
The same thing comes up with every founder I talk to: they're building fast, and they're nervous about what they can't see. QAmeleon is the QA person I'd want on their team.
The proof numbers on the home page come from a real startup where I was the part-time QA person. QAmeleon did the reviews. I checked its work.
What it actually is
QAmeleon is a tool your developer sets up once, in about ten minutes. You connect four things: your code, your app, a few test accounts and your board.
From then on it:
- checks your app every weekday morning and after every deploy
- works through the tickets waiting for QA and tests each one on your dev app
- raises what it finds on your board, with steps so a developer can see it for themselves
- learns your app as it goes, so the morning checks grow over time
It runs on your own setup, using your own AI key. Your tickets and your pages never pass through us.
How I check it's any good
A tool that tests your app should be tested hard itself. These checks run all the time, not once.
An app full of planted bugs
I keep a small practice app with bugs hidden in it on purpose, each one a type of bug found for real on a live product. Some of those bugs are kept back and never used while tuning QAmeleon, so it can't just learn the answers.
A release that gets worse doesn't ship
Every new version is scored on that practice app before it goes out. If it catches fewer bugs, or raises more false alarms, the release is blocked. That has already happened, and the version was held back until it was fixed.
Tickets written to trick it
I plant tickets designed to fool it: hidden instructions telling it to pass a ticket, disguised text, fake messages pretending to be the system, requests to send data somewhere it shouldn't. It has to stop each one before any review starts, and still review the honest tickets that only look suspicious.
Setting it up like a stranger would
I set it up on my own business from the invite email, on an ordinary laptop, and wrote down every snag a newcomer would hit. All 15 were fixed in the next release.
A real product, with real people checking the results
In its first week on a live marketplace, the development lead accepted all 13 of its first findings. None were rejected and none were duplicates.
What it won't do
Some things QAmeleon refuses, however the ticket is worded:
- It won't pass a ticket it couldn't test. No steps, or steps it couldn't finish, never counts as a pass.
- It won't take orders from a ticket. Tickets are something to test, not instructions. If one tries to tell QAmeleon what to do, the review stops and a person decides.
- It won't post security problems on its own. Anything about security, or anything serious, waits for a person to look first.
- It won't spend your money or contact your customers. Checkout, billing, sending, inviting and publishing are always blocked.
- It won't leave a mess. It only creates clearly named test records, deletes them afterwards, and tells you nothing was left behind.
These rules are built in. You can add to them, but you can't switch them off.
What it can't do yet
I'd rather you hear these from me.
- It isn't a replacement for every human check. It's good at the checks people skip. Judgement calls about your product, your brand or your customers still need you.
- The track record is short. The numbers on the home page come from one startup so far. More pilots are starting now.
- It's only as good as what it can reach. With a few test accounts and some realistic data it does far more than without them. It will tell you what to add next.
- Its trick detection isn't perfect. It catches the common tricks and many disguised ones, but a new wording, or a language it doesn't know, could still get past. When that happens, the other rules above and a person's approval are what remain.
- It's built part-time, by one person. That means early teams get me directly, and it also means things move at a steady pace, not overnight.
Questions
Hard questions are welcome, and so are corrections. Email hello@qameleon.ai and you'll get a reply from me.