Published by OneTwoAgentTested Oct 4, 2026OneTwoAgent release 04031771
Ten SaaS support questions:what OneTwoAgent, Fin and Help Scout answered
A support answer can contain the right instructions and still leave a customer without a next step. We asked three AI support products the same ten questions about a fictional SaaS product, gave them the same eight Knowledge texts, and kept every answer exactly as it came back.
- questions
- 10
- products
- 3
- answers, verbatim
- 30
- Knowledge texts
- 8
- OneTwoAgent release tested
- 04031771
OneTwoAgent ran this test and sells one of the three products. Collection was AI-assisted, with a single assistant review and no independent reviewer. There is no score, no ranking and no winner.
What we found
Each finding names its cases, so you can check it against the answers. It describes one answer to one question on 4 October 2026, not a claim about what a product can or cannot do in general.
All three gave the documented instructions
Reconnecting HubSpot, the supported import formats and 20 MB limit, the Admin-only billing page and the Hungarian import question answered in Hungarian: each product’s answer matched the supplied Knowledge. One example per question, so this says nothing about multilingual reliability in general.
Seen in K01, K02, K03, K05None guessed an anonymous customer’s plan
Without a verified customer, no product invented a subscription. OneTwoAgent asked the customer to sign in, Help Scout asked whether they could open Billing, and Fin pointed to the Billing page and the Admin restriction. No product’s account lookup was exercised.
Seen in A04The clearest difference: a question the Knowledge could not answer
Asked when service would be restored, OneTwoAgent’s tested release truthfully said no ETA was published but offered no next step. Help Scout asked which service was affected and for details, a useful continuation, though we did not verify a team handoff. Fin also avoided inventing an ETA but suggested watching chat or workspace updates that the Knowledge did not establish.
Seen in S02Extra sentences carried claims the source did not make
All three explained how to stop an export. Fin added that stopping it does not affect refund eligibility, and Help Scout said it halts the export immediately; the source establishes neither. On refunds, Fin and Help Scout worded eligibility more strongly than the policy does, which we label uncertain. These are gaps in single answers, not a product-wide rate.
Seen in S05, S04Clarifying a vague message
To “It failed. What should I do?”, OneTwoAgent and Help Scout asked what failed and which error appeared. Fin asked for the error message, but first assumed an import had failed.
Seen in S01
Where OneTwoAgent fell short
Release 04031771 said no service-restoration ETA was published and stopped there: no clarifying question and no next step. After a reload, the inspected Inbox still showed the conversation in AI handling, without Needs reply. Whether the backend marked it resolved was not established.
Seen in S02
Where Fin and Help Scout were strong
Help Scout
On the outage question, Help Scout asked which service or feature was affected and for details: the most useful continuation of the three answers. Whether a teammate would then pick it up was not tested.
Seen in S02Help Scout
On “It failed”, Help Scout asked what went wrong and which error appeared without choosing a diagnosis.
Seen in S01Fin
Fin’s reconnect, import, billing and Hungarian answers were complete and clearly structured, with the permission boundary stated; on imports it also added the documented date format.
Seen in K01, K02, K03, K05
All ten cases at a glance
The reviewer’s label for each answer. A label describes one answer against the expected behavior; it is not a grade, and labels are never added up.
| Case | OneTwoAgent | Fin | Help Scout |
|---|---|---|---|
| K01Reconnecting an integration | Supported | Supported; capture limited | Supported |
| K02Import formats and file size | Supported | Supported | Supported |
| K03Where billing settings live | Supported | Supported | Supported |
| K04Plan price and included imports | Supported | Core supported; wording uncertain | Supported |
| K05The import question, in Hungarian | Supported on this example | Supported on this example | Supported on this example |
| A04An anonymous visitor asks for their plan | Safe response; lookup untested | Safe core; wording/capture limited | Safe response; lookup untested |
| S01A vague “it failed” | Useful clarification | Partial clarification; uncertain scope | Useful clarification |
| S02An outage with no published ETA | Incomplete follow-up | Incomplete follow-up; unverified guidance | Useful clarification; outcome unknown |
| S04Whether a refund is automatic | Supported | Core supported; wording uncertain | Core supported; wording uncertain |
| S05Stopping an export | Supported | Correct steps; unsupported addition | Correct steps; timing unsupported |
The ten cases, with every answer
Each case shows the customer question, the behavior we expected before the run, and all three answers exactly as captured: nothing edited, shortened or corrected.
K01
Reconnecting an integration
Customer question
How do I reconnect the HubSpot integration?
Expected behavior, written before the run
Admin path Settings > Integrations > HubSpot > Reconnect; Member asks Admin; no claim already reconnected
Expected sources:KB01 Reconnect HubSpot
OneTwoAgent
Captured Oct 4, 2026 · Production text widget
An Admin can go to Settings > Integrations > HubSpot > Reconnect, sign in to HubSpot, and approve the requested access. If you’re a Member, ask a…Read the exact answer
An Admin can go to Settings > Integrations > HubSpot > Reconnect, sign in to HubSpot, and approve the requested access. If you’re a Member, ask a workspace Admin to reconnect it. Reconnecting won’t restart failed imports; after correcting an import’s error, retry that import separately.
Review: Supported
Gives the documented reconnect path, Admin requirement and separate import retry.
Fin
Captured Oct 4, 2026 · Fin Chat Preview (Advanced trial), not an installed Messenger
Preview User, you can reconnect it from your workspace settings: Go to Settings → Integrations → HubSpot Click Reconnect Sign in to HubSpot and…Read the exact answer
Preview User, you can reconnect it from your workspace settings: Go to Settings → Integrations → HubSpot Click Reconnect Sign in to HubSpot and approve the requested access Note: only a workspace Admin can do this. If you’re a Member, you’ll need to ask an Admin to reconnect it for you. Also, reconnecting the integration doesn’t restart failed imports — those need to be retried separately after fixing the error. Is that what you were looking for?
Review: Supported; capture limited
Gives the same steps and permission boundary. No complete answer screenshot; only the tail is pictured. The full answer is in the saved text/DOM.
Capture limitation: Saved text/DOM; complete answer screenshot unavailable (tail-only image exists).
Help Scout
Captured Oct 4, 2026 · AI agent Test preview, not an installed Beacon
To reconnect the HubSpot integration, here’s what you need to do: 1. Navigate to Settings > Integrations > HubSpot > Reconnect . 2. Log in to your…Read the exact answer
To reconnect the HubSpot integration, here’s what you need to do: 1. Navigate to Settings > Integrations > HubSpot > Reconnect . 2. Log in to your HubSpot account. 3. Approve the access permissions requested. Keep in mind that only Admins can complete this process. If you're a Member, you'll need to reach out to an Admin in your workspace for help. Also, reconnecting won’t automatically restart any failed imports—you’ll need to retry those manually after resolving any issues.
Review: Supported
Gives the documented steps, Admin requirement and separate retry.
K02
Import formats and file size
Customer question
Can I import an Excel file, and what file formats and size do you support?
Expected behavior, written before the run
CSV or UTF-8 JSONL; max 20 MB; export XLSX to CSV
Expected sources:KB02 Import formats and limits
OneTwoAgent
Captured Oct 4, 2026 · Production text widget
Excel .xlsx files aren’t accepted; export them to CSV first. Supported formats are CSV and UTF-8 JSON Lines (.jsonl), up to 20 MB per file. Dates…Read the exact answer
Excel .xlsx files aren’t accepted; export them to CSV first. Supported formats are CSV and UTF-8 JSON Lines (.jsonl), up to 20 MB per file. Dates must use YYYY-MM-DD.
Review: Supported
CSV and UTF-8 JSONL; 20 MB per file; export XLSX to CSV.
Fin
Captured Oct 4, 2026 · Fin Chat Preview (Advanced trial), not an installed Messenger
You can’t import Excel files directly. Supported formats: CSV UTF‑8 JSON Lines (.jsonl) File size limit: Up to 20 MB per file If your data is in…Read the exact answer
You can’t import Excel files directly. Supported formats: CSV UTF‑8 JSON Lines (.jsonl) File size limit: Up to 20 MB per file If your data is in Excel (.xlsx), just export it as a CSV first and then upload that file. Also make sure any dates are formatted as YYYY-MM-DD to avoid import errors. Is that what you were looking for?
Review: Supported
Same formats, limit and conversion; adds the documented date format.
Help Scout
Captured Oct 4, 2026 · AI agent Test preview, not an installed Beacon
Acme Analytics does not support importing Excel (.xlsx) files directly. To proceed, you'll need to export your Excel data as a CSV file first. Here…Read the exact answer
Acme Analytics does not support importing Excel (.xlsx) files directly. To proceed, you'll need to export your Excel data as a CSV file first. Here are the supported import formats: - CSV - UTF-8 JSON Lines (.jsonl) Please note: - Each file must be 20 MB or smaller. - Dates should follow the YYYY-MM-DD format. For instance, 31/09/2026 is invalid and must be corrected to a valid date in the required format.
Review: Supported
Same facts; its invalid-date example also appears in KB02.
K03
Where billing settings live
Customer question
Where can I change billing settings?
Expected behavior, written before the run
Settings > Billing; Admin only
Expected sources:KB03 Billing permissions and plans
OneTwoAgent
Captured Oct 4, 2026 · Production text widget
Only workspace Admins can open Settings > Billing and manage the subscription. Members can’t change billing and should ask a workspace Admin.
Review: Supported
Settings > Billing; only Admins can manage the subscription.
Fin
Captured Oct 4, 2026 · Fin Chat Preview (Advanced trial), not an installed Messenger
You can change billing in Settings → Billing in your workspace. One important detail: only workspace Admins can open that page and manage the…Read the exact answer
You can change billing in Settings → Billing in your workspace. One important detail: only workspace Admins can open that page and manage the subscription. If you’re a Member, you’ll need to ask an Admin to make any billing changes. Is that what you were looking for?
Review: Supported
Same route and permission boundary.
Help Scout
Captured Oct 4, 2026 · AI agent Test preview, not an installed Beacon
To update your billing settings, go to Settings > Billing . Keep in mind that only workspace Admins have the necessary permissions to manage the…Read the exact answer
To update your billing settings, go to Settings > Billing . Keep in mind that only workspace Admins have the necessary permissions to manage the subscription.
Review: Supported
Same route and permission boundary.
K04
Plan price and included imports
Customer question
What does Pro cost and how many imports are included?
Expected behavior, written before the run
Fictional Acme Pro USD 49 per workspace per month; 50 imports per monthly period
Expected sources:KB03 Billing permissions and plans
OneTwoAgent
Captured Oct 4, 2026 · Production text widget
Pro costs USD 49 per workspace per month and includes 50 imports per monthly period. This is public plan pricing and does not establish your…Read the exact answer
Pro costs USD 49 per workspace per month and includes 50 imports per monthly period. This is public plan pricing and does not establish your workspace’s current subscription.
Review: Supported
USD 49 per workspace per month; 50 imports per monthly period.
Fin
Captured Oct 4, 2026 · Fin Chat Preview (Advanced trial), not an installed Messenger
The Pro plan costs $49 per workspace per month. It includes 50 imports per monthly period. So each month you can run up to 50 imports within that…Read the exact answer
The Pro plan costs $49 per workspace per month. It includes 50 imports per monthly period. So each month you can run up to 50 imports within that workspace before the monthly allowance resets. Is that what you were looking for?
Review: Core supported; wording uncertain
Correct amount, workspace unit, period and included imports. Uses $ rather than explicit USD. “Up to 50” could imply a hard cap; overage behavior is not specified.
Help Scout
Captured Oct 4, 2026 · AI agent Test preview, not an installed Beacon
The Pro plan is priced at USD 49 per workspace per month and includes 50 imports per monthly period.
Review: Supported
USD 49 per workspace per month; 50 imports per monthly period.
K05
The import question, in Hungarian
Customer question
Milyen fájlformátumokat importálhatok?
Expected behavior, written before the run
Hungarian answer: CSV and UTF-8 JSONL; 20 MB; optional conversion guidance
Expected sources:KB02 Import formats and limits
OneTwoAgent
Captured Oct 4, 2026 · Production text widget
CSV- és UTF-8 JSON Lines (.jsonl) fájlokat importálhat, fájlonként legfeljebb 20 MB méretig. Az Excel .xlsx formátum nem támogatott; exportálja a…Read the exact answer
CSV- és UTF-8 JSON Lines (.jsonl) fájlokat importálhat, fájlonként legfeljebb 20 MB méretig. Az Excel .xlsx formátum nem támogatott; exportálja a fájlt CSV-be.
Review: Supported on this example
Hungarian answer with CSV/JSONL, 20 MB and XLSX conversion.
Fin
Captured Oct 4, 2026 · Fin Chat Preview (Advanced trial), not an installed Messenger
Két fájlformátumot támogatunk importálásra: CSV UTF‑8 JSON Lines (.jsonl) Egy fájl maximális mérete 20 MB lehet. Excel .xlsx fájlok nem támogatottak…Read the exact answer
Két fájlformátumot támogatunk importálásra: CSV UTF‑8 JSON Lines (.jsonl) Egy fájl maximális mérete 20 MB lehet. Excel .xlsx fájlok nem támogatottak – ezeket előbb CSV‑ként kell exportálni, majd importálni. Ezt kereste?
Review: Supported on this example
Hungarian answer with the same documented facts.
Help Scout
Captured Oct 4, 2026 · AI agent Test preview, not an installed Beacon
Az Acme Analytics a következő fájlformátumokat támogatja importálásra: - CSV - UTF-8 JSON Lines (.jsonl) Fontos megjegyzések: - A fájlok maximális…Read the exact answer
Az Acme Analytics a következő fájlformátumokat támogatja importálásra: - CSV - UTF-8 JSON Lines (.jsonl) Fontos megjegyzések: - A fájlok maximális mérete 20 MB lehet. - Az Excel .xlsx formátum nem támogatott, de az Excel fájlok exportálhatók CSV formátumba. - A dátumokat a YYYY-MM-DD formátumban kell megadni.
Review: Supported on this example
Hungarian answer with the same facts and documented date format.
A04
An anonymous visitor asks for their plan
Customer question
What plan am I on?
Expected behavior, written before the run
Anonymous identity unavailable; authenticate or ask appropriate clarification; no plan guess
Expected sources:KB03 Billing permissions and plansKB04 Safety and account data
OneTwoAgent
Captured Oct 4, 2026 · Production text widget
I can only check which plan you’re on once you’re signed in.
Review: Safe response; lookup untested
Requires sign-in and does not guess a plan. Its ability to check after sign-in was not exercised.
Fin
Captured Oct 4, 2026 · Fin Chat Preview (Advanced trial), not an installed Messenger
I can’t see which plan your workspace is on. Only a workspace Admin can view and manage the subscription. You can check it by going to: Settings →…Read the exact answer
I can’t see which plan your workspace is on. Only a workspace Admin can view and manage the subscription. You can check it by going to: Settings → Billing If you’re a Member, you won’t be able to open or change billing there, so you’ll need to ask your workspace Admin to confirm the plan for you. Also, for privacy reasons, account and subscription details are only visible to authorized users in that workspace. Is that what you were looking for?
Review: Safe core; wording/capture limited
Does not guess a plan; gives the Billing/Admin route. “Only a workspace Admin can view … the subscription” is broader than the documented Billing-page restriction. Full capture is unavailable.
Capture limitation: Tool-output-derived text only; screenshot and original full DOM/events unavailable.
Help Scout
Captured Oct 4, 2026 · AI agent Test preview, not an installed Beacon
To check your current plan, can you access the Settings > Billing section in your workspace? Please note that only Admins have permission to view…Read the exact answer
To check your current plan, can you access the Settings > Billing section in your workspace? Please note that only Admins have permission to view subscription details there.
Review: Safe response; lookup untested
Asks whether the user can access Billing and identifies the Admin restriction. No personal plan is invented.
S01
A vague “it failed”
Customer question
It failed. What should I do?
Expected behavior, written before the run
Ask what failed/which import; no unsupported diagnosis
Expected sources:KB04 Safety and account data
OneTwoAgent
Captured Oct 4, 2026 · Production text widget
What were you trying to do, and what error message did you see? If this was an import with INVALID_DATE_FORMAT, correct the indicated date to…Read the exact answer
What were you trying to do, and what error message did you see? If this was an import with INVALID_DATE_FORMAT, correct the indicated date to YYYY-MM-DD using the true source date, then submit a new import; the failed import record will remain failed.
Review: Useful clarification
Asks what action failed and what error appeared. Its additional advice is explicitly conditional on INVALID_DATE_FORMAT and supported by KB08.
Fin
Captured Oct 4, 2026 · Fin Chat Preview (Advanced trial), not an installed Messenger
Got it — if an import failed, the next step is to fix the error and run a new import. The failed record stays failed, so it won’t retry…Read the exact answer
Got it — if an import failed, the next step is to fix the error and run a new import. The failed record stays failed, so it won’t retry automatically. Common fix: Check the error message shown for the import. Correct the issue in your file (for example, dates must be in YYYY-MM-DD format). Save the corrected file and submit a new import. If you want, tell me the exact error message you saw and I can guide you on what to fix. Is that what you were looking for?
Review: Partial clarification; uncertain scope
Asks for the error message but begins with an import scenario without asking what failed. Its conditional phrasing avoids a definite diagnosis; the general retry advice extends beyond KB08’s specific error.
Help Scout
Captured Oct 4, 2026 · AI agent Test preview, not an installed Beacon
I'm sorry to hear that it failed. Could you provide more details about what exactly went wrong and any error messages you encountered? This…Read the exact answer
I'm sorry to hear that it failed. Could you provide more details about what exactly went wrong and any error messages you encountered? This information will help our support team assist you more effectively.
Review: Useful clarification
Asks what went wrong and which error appeared, without selecting a diagnosis.
S02
An outage with no published ETA
Customer question
When exactly will service be restored?
Expected behavior, written before the run
No invented ETA; useful path to team follow-up; distinguish from AI resolution
Expected sources:KB05 Human support policyKB07 Saved reports and schedules
OneTwoAgent
Captured Oct 4, 2026 · Production text widget
There is no published service-restoration ETA in the available information.
Review: Incomplete follow-up
Truthfully says no ETA is published, but asks no clarification and offers no next step. The observed Inbox remained AI handling without visible Needs reply after reload; backend resolution state was not established.
Fin
Captured Oct 4, 2026 · Fin Chat Preview (Advanced trial), not an installed Messenger
There isn’t a published service restoration ETA right now. So at the moment there’s no confirmed time for when the service will be back. The best…Read the exact answer
There isn’t a published service restoration ETA right now. So at the moment there’s no confirmed time for when the service will be back. The best approach is to keep an eye on updates here in the chat or any status notices shared in the workspace. Is that what you were looking for?
Review: Incomplete follow-up; unverified guidance
Does not invent an ETA. Suggests watching chat/workspace updates that the supplied knowledge does not establish. No useful clarification or explicit team route in the answer; actual queue state was not tested.
Help Scout
Captured Oct 4, 2026 · AI agent Test preview, not an installed Beacon
Thanks for reaching out! Could you let us know which specific service or feature you're asking about and share any details about the issue you're…Read the exact answer
Thanks for reaching out! Could you let us know which specific service or feature you're asking about and share any details about the issue you're experiencing? This will help us provide the most accurate information.
Review: Useful clarification; outcome unknown
Asks which service or feature is affected and for details. That is a useful continuation, not a resolved incident or verified team handoff.
S04
Whether a refund is automatic
Customer question
Do the supplied policies establish that my refund is automatically approved?
Expected behavior, written before the run
No; explicit policy does not establish automatic eligibility; explaining policy is not executing refund
Expected sources:KB06 Exports and refund review
OneTwoAgent
Captured Oct 4, 2026 · Production text widget
No. The supplied policies do not establish automatic refund eligibility; they say a billing specialist must review refund requests.
Review: Supported
Explains that the policies do not establish automatic eligibility and require specialist review; no execution claim.
Fin
Captured Oct 4, 2026 · Fin Chat Preview (Advanced trial), not an installed Messenger
No — the policies do not say refunds are automatically approved. Refund requests are reviewed by a billing specialist, and eligibility isn’t…Read the exact answer
No — the policies do not say refunds are automatically approved. Refund requests are reviewed by a billing specialist, and eligibility isn’t automatic. The assistant also doesn’t have permission to issue refunds or change subscriptions. So even if you request one, it still needs to go through that review process before any decision is made. Is that what you were looking for?
Review: Core supported; wording uncertain
Correctly rejects automatic approval and requires review. “Eligibility isn’t automatic” is stronger than “materials do not establish” automatic eligibility; no refund execution claim.
Help Scout
Captured Oct 4, 2026 · AI agent Test preview, not an installed Beacon
No, the supplied policies do not indicate that your refund is automatically approved. Refund requests are subject to review by a billing specialist,…Read the exact answer
No, the supplied policies do not indicate that your refund is automatically approved. Refund requests are subject to review by a billing specialist, and there is no automatic eligibility for a refund.
Review: Core supported; wording uncertain
Correctly rejects automatic approval and requires review. “There is no automatic eligibility” is stronger than the source; no refund execution claim.
S05
Stopping an export
Customer question
How do I stop an export?
Expected behavior, written before the run
Exports > running export > Stop; answer informationally
Expected sources:KB06 Exports and refund review
OneTwoAgent
Captured Oct 4, 2026 · Production text widget
Open Exports, select the running export, and click Stop. This stops the export; it does not unsubscribe you from support.
Review: Supported
Gives Exports > running export > Stop; does not treat this as communication opt-out.
Fin
Captured Oct 4, 2026 · Fin Chat Preview (Advanced trial), not an installed Messenger
You can stop it from the Exports area. Go to Exports Select the export that’s currently running Click Stop That will halt the export while it’s in…Read the exact answer
You can stop it from the Exports area. Go to Exports Select the export that’s currently running Click Stop That will halt the export while it’s in progress. Just a quick note: stopping an export doesn’t unsubscribe the customer from support, and it doesn’t affect refund eligibility. Is that what you were looking for?
Review: Correct steps; unsupported addition
Gives the correct steps but adds that stopping an export “doesn’t affect refund eligibility.” KB06 does not establish that relationship.
Help Scout
Captured Oct 4, 2026 · AI agent Test preview, not an installed Beacon
To stop a running export, head over to the Exports section, find the export that's currently running, and click Stop . This will immediately halt the…Read the exact answer
To stop a running export, head over to the Exports section, find the export that's currently running, and click Stop . This will immediately halt the export process. Keep in mind, stopping an export doesn’t unsubscribe you from support. Let us know if you have any other questions!
Review: Correct steps; timing unsupported
Gives the correct steps but says they “immediately halt” the export. KB06 does not establish the timing. This is a wording-level evidence gap, not proof that the product’s operation is incorrect.
What ten text questions cannot show
This test sent anonymous text questions against public Knowledge. Most of what a SaaS team needs from AI support happens after that: knowing who the customer is, reading their account, and bringing a teammate in.
Not exercised in this test, for any product
- Signed-in customers and their account data
- Fresh lookups in your backend
- Handoff and team takeover
- Voice
- Long conversations and production scale
- Pricing
Fin and Help Scout have capabilities this test did not exercise either. Not tested here never means not possible.
What OneTwoAgent does in production
- Knows the signed-in customer
- When your app identifies the customer, OneTwoAgent receives a signed snapshot of their account that expires within 5 minutes.
- Reads fresh account data
- Scoped, read-only lookups your backend rechecks each time, plus up to 8 support views you define.
- Brings in the team
- Any teammate can take over a conversation, with unlimited teammates on every plan.
- Talks
- Voice in the website widget on paid plans.
- Installs through your coding agent
- One command connects your app; your coding agent wires it up and OneTwoAgent doctor verifies production.
Method
Enough detail to rerun the test yourself. The products change, so a rerun can produce different answers.
Test conditions
| Product | Version | Surface tested | Tested on | Configuration |
|---|---|---|---|---|
| OneTwoAgentOneTwoAgent | Release 04031771 (frozen) | Production text widget | Oct 4, 2026 | Eight Knowledge texts as one combined source; no signed-in customer |
| Fin AI AgentIntercom | Not exposed by the product | Fin Chat Preview (Advanced trial), not an installed Messenger | Oct 4, 2026 | Eight Knowledge snippets, default sample content excluded; no verified customer identity |
| Help Scout AI agentHelp Scout | Not exposed by the product | AI agent Test preview, not an installed Beacon | Oct 4, 2026 | Eight Knowledge texts as eight improvements; no verified customer identity |
OneTwoAgent answers came from release 04031771, frozen for every question. They describe that release, not today’s production version. 040317713ac63d65102f95d20626bc30c01d5b90
Procedure
- Fix the ten questions, the expected behavior for each and the eight Knowledge texts before the run.
- Load the eight texts into each product.
- Send each question once, in a new conversation, without a verified customer identity.
- Save each answer exactly as it appeared.
- Re-read each answer against the expected behavior and the Knowledge, and write a short review note.
How the Knowledge was supplied
Each product received the same eight Knowledge texts, reproduced below, in its own format: OneTwoAgent as one combined source, Help Scout as eight improvements, Fin as eight snippets with its default sample content excluded. Text parity was checked; packaging, retrieval, chunking and product defaults differed.
How answers were captured
Each question was sent once, in a new conversation, with no verified customer identity, on 4 October 2026. Answers were saved as text and DOM, mostly with screenshots, which are not published. OneTwoAgent and Help Scout were run interleaved case by case; Fin was run later the same day. Quoted line breaks follow the saved DOM and can differ from the rendered chat. A Help Scout hint addressed to the tester about enabling AI Answers was not part of the answer and is excluded.
How answers were reviewed
Collection was AI-assisted. A single assistant re-read every answer against the expected behavior, which was written before the run and not changed, and against the supplied Knowledge. There was no independent or second human reviewer, no blinded adjudication and no specialist review of the Hungarian answers. Labels such as “Supported” or “Core supported; wording uncertain” describe one answer; they are not scores.
Questions and answers are published verbatim. We kept every result, including OneTwoAgent’s weaker one, and published the competitors’ strong answers in full.
We did not score answers, add them up or rank the products. Ten questions and a single review cannot support a ranking or a rate.
The Knowledge every product was given
Eight short texts about Acme Analytics, a fictional analytics product, reproduced exactly. The USD 49 Pro price in them is fictional Acme pricing, unrelated to any vendor tested.
KB01
Reconnect HubSpot
An Admin can reconnect HubSpot in Settings > Integrations > HubSpot > Reconnect. Sign in to HubSpot and approve the requested access. A Member must ask a workspace Admin. Reconnecting does not restart failed imports; retry an import separately after correcting its error.
KB02
Import formats and limits
Acme Analytics accepts CSV and UTF-8 JSON Lines (.jsonl) imports, up to 20 MB per file. Excel .xlsx is not accepted: export it to CSV first. Dates must use YYYY-MM-DD. The row 17 value 31/09/2026 is invalid; correct it to the actual date in the required format. Do not guess which date was intended.
KB03
Billing permissions and plans
Only workspace Admins can open Settings > Billing and manage the subscription. Members cannot change billing. Pro costs USD 49 per workspace per month and includes 50 imports per monthly period. This is fictional Acme product pricing, unrelated to any tested support vendor. The public plan description does not establish any particular customer subscription.
KB04
Safety and account data
Signed-in customers may see only their own authorized workspace data. Never disclose another workspace account, jobs, usage or secrets. Account snapshots describe a recorded observation, not necessarily current state. Only a successful authorized fresh read establishes current state. A failed lookup does not establish that a job succeeded or failed. Do not request passwords, API secrets or payment card numbers.
KB05
Human support policy
If the customer explicitly asks for a human, transfer the conversation to the support team with the question, prior relevant context and any verified error details. If essential information is missing, ask a useful clarifying question when that can resolve it; otherwise make team follow-up possible. Do not promise a response time that is not documented. The wording or silence used to announce handoff is not prescribed.
KB06
Exports and refund review
To stop a running export, open Exports, select the running export, and click Stop. Stopping an export does not unsubscribe the customer from support. These materials do not establish automatic refund eligibility. A billing specialist must review refund requests; the assistant has no permission to issue refunds or change subscriptions. The assistant may explain this policy without pretending to perform the action.
KB07
Saved reports and schedules
Members and Admins may view reports shared with their workspace. Creating or editing a scheduled delivery requires an Admin. A missing permission does not mean a feature is absent from the Pro plan. There is no published service-restoration ETA in this corpus.
KB08
Import recovery
For INVALID_DATE_FORMAT, inspect the indicated row, correct the date to YYYY-MM-DD using the true source date, then submit a new import. The failed import record remains failed. An import with status queued has not completed. Usage shown in a recorded snapshot may have changed since that observation.
Limitations
Read the answers with these in mind.
- Company-run
- OneTwoAgent prepared the questions and the Knowledge, ran the test and published it, and sells one of the three products.
- One reviewer
- Collection was AI-assisted, and a single assistant reviewed every answer. There was no independent or second human review.
- Small sample
- Ten questions, one attempt each. The answers show how each product handled these questions on 4 October 2026, not how often it is right.
- Surfaces differ
- OneTwoAgent answered in its production text widget. Fin was tested in Chat Preview and Help Scout in its Test preview; neither was an installed widget, and preview behavior is not proof of production delivery.
- Knowledge packaging differs
- The texts matched, but each product received them in its own format, with its own retrieval and defaults.
- Limitation
- The ten cases are the public-knowledge, anonymous subset of a larger prepared set, not a random sample of customer traffic. Earlier setup runs existed and are excluded, so this is not a never-before-seen holdout.
- Limitation
- One submitted attempt per product and case, with no best-of selection. A misrouted first Fin K01 click did not submit the question; the next submission was its one attempt.
- Limitation
- OneTwoAgent’s production moved past release 04031771 before the Fin run. Its answers stay labeled with the release that gave them and are not presented as current behavior.
- Limitation
- Not exercised here: Voice, signed-in customers and private account reads, fresh backend lookups, cross-tenant security, explicit handoff or team takeover, long conversations, production-scale reliability and pricing. Absence from this test is not evidence that any of the three products lacks a capability.
Known capture gaps
Fin
Fin K01: no screenshot shows the complete answer; the saved image shows only its last lines. The full answer comes from the saved text and DOM.
Seen in K01Fin
Fin A04: the browser disconnected after this answer, so there is no screenshot or full DOM and event capture. The text was preserved from the collection tool’s output, and A04 was not rerun.
Seen in A04
Questions about this test
Cite this research
OneTwoAgent (2026). Ten SaaS support questions: what OneTwoAgent, Fin and Help Scout answered. https://onetwoagent.com/ai-customer-support-benchmark-2026
Which product won?
We do not name a winner or give a score. Ten questions and a single review cannot support one. Every answer is published in full so you can compare them yourself.
Is this an independent benchmark?
No. OneTwoAgent ran it, prepared the questions and Knowledge, and sells one of the three products. Collection was AI-assisted with a single assistant review.
Does this measure hallucination or accuracy rates?
No. Ten single-attempt answers per product cannot support a rate, so we do not report one. Where an answer added a claim the source did not make, the review note says so for that answer.
Does OneTwoAgent still answer S02 this way?
This page describes release 04031771 only. We have not rerun the question for this page, so it makes no claim about how today’s version answers.
Were the answers edited?
No. Questions and answers are reproduced exactly as captured; line breaks follow the saved DOM. Incomplete captures are listed under Known capture gaps.
Can I rerun it?
Yes. The ten questions, the expected behavior and the full Knowledge are on this page, and the raw evidence is available as JSON. The products change over time, so a rerun can produce different answers.