Building a Real-Time Romance Scam Detection System with Machine Learning

A romance scam does not usually arrive with a flashing red warning sign.

It may begin with a well-written profile, a friendly message and a conversation that seems completely ordinary. The account might have photographs, shared interests and just enough personal detail to feel convincing. The problem only becomes visible later, when the same profile starts repeating a script, pushing someone to move the conversation elsewhere or creating a sense of urgency around money.

That is what makes these cases difficult to handle with technology alone. Most of the early signals look harmless in isolation. A new account is not suspicious by default. Nor is someone sending a heartfelt message, using a VPN or talking to several people at once. The risk appears in the pattern.

For developers building trust-and-safety systems, that means the job is less about finding one perfect “scam detector” and more about connecting small behavioural clues quickly enough to make a difference.

Fraud Detection Is a Streaming Problem

It is tempting to think of account safety as a database task: store profiles, run a nightly check and flag anything unusual. In reality, that approach is usually too slow.

A profile that begins sending near-identical messages to twenty people does not need to wait until tomorrow to be assessed. The same applies to an account that suddenly changes devices, adds external contact details and receives multiple reports in a short period. Each action should become an event that the platform can process while it is happening.

A basic real-time flow might look like this:

User event → stream → feature service → risk model → action

The events can include account registration, logins, messages, reports, blocks, profile updates and verification outcomes. A streaming platform such as Kafka, Kinesis or Pub/Sub collects them. From there, a lightweight service enriches the event with account history and sends a set of features to a risk-scoring system.

If you are building this on AWS, Stackademic’s guide to real-time fraud intelligence is a useful starting point for thinking about event-driven architecture, low-latency processing and the operational side of fraud prevention.

The stack itself is not the interesting part, though. The difficult question is what the system should pay attention to.

Look for Behaviour, Not a “Scammer Profile”

There is no universal scammer profile. Bad actors change photographs, locations, languages and stories constantly. Rules based purely on static profile data tend to become outdated very quickly.

Behaviour is harder to fake consistently.

For example, an account that was created ten minutes ago, sends the same introduction to thirty people and receives no meaningful replies deserves attention. So does an account that shares a device or IP address with several profiles previously removed for abuse.

None of these signals proves wrongdoing. A new user may simply be eager to meet people. A shared IP could be a university network, a hotel or a family home. But when several signals arrive together, the picture becomes clearer.

A useful feature set could include:

• age of the account;
• completeness and verification status of the profile;
• message volume over the previous hour and day;
• number of unique recipients;
• reply, block and report rates;
• frequency of device or location changes;
• similarity between messages;
• number of connected accounts sharing devices, IP addresses or payment data.

The goal is not to build a system that assumes the worst about every new user. It is to give the platform enough context to recognise when activity stops looking organic.

Message Similarity Is Often More Useful Than Keywords

Keyword filters have their place. They can catch contact details, suspicious links or obvious attempts to direct a user to another platform. But scammers rarely depend on one phrase for long. If a word becomes blocked, they change the spelling, use an image or make the request more indirect.

This is where natural language processing becomes useful.

Instead of searching only for a handful of banned words, a model can compare the meaning of messages. It can identify whether one account is repeatedly sending the same story with only minor edits: a supposedly urgent family problem, a sudden travel issue or an unusually polished first message copied across dozens of conversations.

A simple embedding-based check might look like this:

from sklearn.metrics.pairwise import cosine_similarity

def reused_message(new_vector, previous_vectors, threshold=0.91):

    scores = cosine_similarity([new_vector], previous_vectors)[0]

    return max(scores, default=0) > threshold

This does not mean every repeated message should trigger a ban. Plenty of real people recycle their best opener. The useful signal comes when message similarity appears alongside other unusual activity — especially high-volume outreach, a very new account or multiple user reports.

In other words, NLP should help moderators ask better questions, not pretend it can read intent with perfect accuracy.

Graphs Reveal Relationships That Profiles Hide

One suspicious account is useful. A cluster of related accounts is more useful.

Scam operations often reuse infrastructure. The accounts may share devices, IP ranges, contact details, payment methods or a pattern of moving users to the same external service. These connections are easy to miss when every profile is evaluated separately.

Graph analysis helps expose that structure.

You can model accounts, devices, IP addresses, payment instruments and external contact points as nodes. Each login, transaction or message creates an edge between them. Over time, the graph shows whether a supposedly independent account sits close to previously flagged activity.

Even basic graph features can improve a model significantly:

Account A

   ├── Device X

   ├── IP Range 12

   └── External contact Y


Account B

   ├── Device X

   └── External contact Y

If Account B has already been suspended, Account A should not be treated as entirely unknown. It may not deserve an immediate removal, but the relationship gives the platform a strong reason to increase its risk score or require additional verification.

Neo4j, Amazon Neptune and graph-capable data warehouses can all support this kind of work. The right choice depends on scale and team experience. What matters most is that the relationship data is available when the decision is being made, not weeks later in a retrospective report.

Let Rules and Models Do Different Jobs

Machine learning gets most of the attention in fraud detection, but a production system still needs rules.

Some behaviour is easy to define. A new account might be rate-limited before it can message hundreds of people. Public profiles can be prevented from displaying sensitive contact information. A user who receives several credible reports in a short period can be temporarily restricted while the case is reviewed.

These rules are fast, understandable and easy to change. They are also useful when a new abuse pattern appears and there is not yet enough labelled data to retrain a model.

Machine learning is better at evaluating the grey area: the combination of moderate signals that would not justify action on their own.

A simple risk score can combine both approaches:

def risk_score(features, model_probability):

    score = model_probability


    if features["account_age_hours"] < 24:

        score += 0.07


    if features["message_similarity"] > 0.91:

        score += 0.15


    if features["linked_flagged_accounts"] > 1:

        score += 0.22


    if features["reports_last_24h"] >= 3:

        score += 0.28


    return min(score, 1.0)

The final score should lead to proportionate actions. A low-risk account continues normally. A medium-risk account might face a temporary messaging limit or a verification prompt. A high-risk account goes to a review queue, where a moderator can inspect the evidence before making a decision.

That middle layer is important. Not every unusual action should lead to a hard ban.

The Human Reviewer Still Matters

Anyone who has worked with real moderation data knows that context can change everything.

A user sending lots of messages could be a scammer — or someone who joined after a long time away and is simply too enthusiastic. A device shared by multiple accounts might signal organised abuse, but it could also belong to a household. An account discussing money may be trying to manipulate someone, or it may be talking about work, travel or everyday life.

A reviewer needs more than a model score. They need a short explanation of why the case was escalated:

Risk score: 0.81

Reasons:

• Account created 3 hours ago

• 24 highly similar messages sent

• Device linked to two suspended accounts

• Four reports received today

That information lets the reviewer make a faster, more defensible decision. It also improves the model over time. Every confirmed case, cleared account and successful appeal becomes feedback for the next training cycle.

The best systems treat human review as part of the product, not as a manual fallback for when automation fails.

Dating Platforms Should Be Judged Separately From Bad Actors

The existence of online scammers does not make a dating platform itself fraudulent. Dating.com is a legitimate dating service, not a scam. When users search for dating com scam, they are usually looking for reassurance about how the platform handles safety and how they can recognise suspicious behaviour before it becomes a problem.

That distinction matters. The developers and trust-and-safety teams behind a platform are not there to create a perfect world where abuse never happens; no public online service can honestly promise that. Their responsibility is to make bad behaviour harder to carry out, easier to detect and quicker to investigate.

That work is a mix of engineering and people. Automated systems can spot unusual account connections, repeated message patterns and sudden changes in behaviour at a scale no human team could handle alone. Moderators then bring judgement to the cases where a model needs context. Clear reporting tools allow users to contribute to the process too.

It is this layered approach — prevention, detection, moderation and user education — that makes an online dating service more resilient.

Privacy Cannot Be Bolted on Later

There is an obvious tension here. Safety models work best when they have rich behavioural data, but dating conversations and profiles are personal by nature.

The answer is not to collect everything. It is to collect what is necessary, retain it for an appropriate period and keep access tightly controlled. Derived features are often safer to store than raw message content. Internal tools should log who accessed a case and why. Data should be encrypted in transit and at rest, and moderation teams should work with clear policies rather than vague discretion.

Users also need a meaningful path to appeal or verify their identity if an automated system gets something wrong. Fairness is not just a legal or ethical concern; it is how the product earns trust from the people it is meant to protect.

A real-time romance scam detection system is not one model deployed behind an API. It is an ecosystem: event streams, behavioural features, NLP, graph relationships, rules, model scores and human review all working together.

The strongest systems do not punish every odd interaction. They learn to separate harmless awkwardness from patterns that put people at risk. That requires thoughtful engineering, good feedback loops and a willingness to keep improving as scammers change tactics.

For a dating platform, trust is not a feature that ships once. It is something the product has to earn in every conversation.

Comments

Loading comments…