Can phishing models learn without pooling URLs?
A peer-reviewed benchmark tests seven classifiers on 11,430 labelled URLs and simulates federated learning across 20 clients. XGBoost leads centrally, while a feature-weighted neural model stays competitive without moving raw examples—but no real organisations or live traffic were tested.
Editorial responsibility: The Impact of AI Editorial Desk · Report a factual concern
At a glance
- 1Seven architectures were compared on a balanced public dataset of 11,430 URLs and 87 engineered features using ten-fold stratified cross-validation, paired McNemar tests and CPU-only latency measurements.
- 2Central XGBoost led with 97.00% accuracy and 0.79 ms median cached-feature inference. In a separate 80/20 experiment, the federated SAFW-Hybrid averaged 95.4% ± 0.5% over its final five rounds across 20 simulated clients.
- 3The clients, non-IID differences and communication were simulated from one old public benchmark. The experiment provides no formal privacy guarantee, threat model or evidence from real organisations and current phishing traffic.
Research topic
Whether interpretable feature weighting and federated aggregation can classify phishing URLs competitively when raw examples remain within simulated organisational clients
The study compares seven classifiers under one benchmark protocol
Kahkashan Kouser and colleagues evaluated phishing URL detection on the Hannousse–Yahiouche public dataset: 11,430 examples, split evenly between 5,715 phishing and 5,715 legitimate URLs, with 87 numeric features describing lexical, host, content and external signals. The paper compares XGBoost, random forest, support-vector machine, a one-dimensional convolutional network, a bidirectional LSTM, the authors' SHAP-Guided Adaptive Feature Weighting hybrid and a DistilBERT reference case. Central results use ten-fold stratified cross-validation across the full dataset rather than one favourable holdout.
XGBoost led the central comparison with 97.00% accuracy, an area under the receiver-operating curve of 0.9949 and a Matthews correlation coefficient of 0.94. The one-dimensional CNN reached 95.58% accuracy and the SAFW-Hybrid 95.64%. The authors pooled out-of-fold predictions and ran all 21 pairwise McNemar comparisons with continuity correction. That matters because close scores do not automatically represent meaningful differences: for example, the paper found no statistically detectable difference between the SVM and BiLSTM predictions at P = 0.469. The protocol is more careful than a single accuracy table, although it still measures one benchmark rather than future attacks.[1][2]
The Impact Brief · Free
Follow the evidence in security & defence.
Get a five-minute weekday briefing on what changed, why it matters and where the evidence comes from. Choose the topics you care about.
Feature weighting keeps every signal but makes its influence visible
The proposed SAFW layer begins with mean absolute SHAP values from a trained XGBoost model. Those scores initialise a trainable weight for each of the 87 features before a neural feature branch and sequence branch combine their outputs. Unlike feature-selection methods that permanently drop low-ranked inputs, the design retains every feature with a floor so later training can still raise a signal the tree model initially undervalued. The resulting weight vector is intended to give analysts a clearer account of what the model emphasises than a conventional hidden representation.
Interpretability remains conditional. The highest-ranked features include google_index and page_rank, both obtained through external lookups, as well as URL-structure signals such as hostname length and phishing hints. A learned weight is not a causal explanation of why a URL is malicious, and correlated features can substitute for each other. The paper appropriately reports the direction and magnitude of the tree-based SHAP analysis and examines the learned weights across clients, but it does not test whether incident responders make better decisions with those explanations or whether an attacker can deliberately manipulate the most influential features.[1]
The federated result comes from 20 simulated clients
For the federated extension, the researchers created a separate stratified 80/20 split: 9,144 training URLs and 2,286 shared test URLs. They partitioned the training portion across 20 simulated organisations with a Dirichlet procedure intended to create different class mixes. Candidate partitions were rejected unless every client held at least 20 examples from each class, making the realised heterogeneity less extreme than an unconstrained alpha of 0.5. The model then ran ten FedAvg communication rounds. Raw examples were not transmitted in the simulation, but model updates were.
Federated SAFW-Hybrid reached 95.84% accuracy in round ten and averaged 95.4% with a standard deviation of 0.5 percentage points over the final five rounds. Its central counterpart on the identical split reached 95.45%. A paired McNemar test on the shared test URLs produced P = 0.30, so the experiment found no statistically detectable difference between the two error patterns. The reported paired-difference confidence interval ran from minus 1.05 to plus 0.22 percentage points, which did not fit wholly inside the authors' proposed plus-or-minus 0.5-point equivalence margin. The responsible conclusion is therefore competitive performance under this simulation, not formal equivalence.[1]
A purpose-built tree method avoids the naive federated loss
A naive federated XGBoost ensemble lost 4.20 points relative to central XGBoost, reaching 92.34% on the shared split. A cyclic federated boosting design recovered most of that loss, reporting 95.14% in the primary run and between about 94.5% and 95.1% across tested settings. This is a valuable comparator: the neural SAFW model was not uniquely capable of distributed learning. Its advantage is the combination of competitive classification with an explicit, shared feature-weight representation; the better engineering choice depends on whether organisations value that representation, communication pattern and model family.
The study also tested multiple seeds and two nominal heterogeneity settings, reporting weaker local-client accuracy as low as 75.81% in some runs even while the aggregate test score stayed near 95%. That gap is practically important. A global average can conceal a participant whose local mix is poorly served, exactly the situation federated learning is meant to address. Real deployments would need per-client calibration, minimum-performance rules and investigation of subgroup failures rather than assuming a strong pooled test result protects every member.[1]
Fast inference assumes two outside signals are already available
The authors benchmarked CPU inference under a common procedure. XGBoost had a median latency of 0.79 milliseconds, a 95th percentile of 3.70 ms and a 99th percentile of 12.01 ms when all features were ready, fitting a 20-ms browser-extension target. But google_index and page_rank require external API lookups. A cold request added an estimated 100–500 ms, two to three orders of magnitude more than the classifier itself. The headline latency is therefore a cached-feature result, not end-to-end speed for a newly observed URL.
Removing those two features reduced central XGBoost from 97.00% to 96.09% under cross-validation and left the broad federated conclusions intact. Robustness under severe class imbalance deteriorated much more, however, with the F1 score at the most extreme ratio varying by 35.6 percentage points across architectures. That exposes a deployment trade-off: external reputation signals improve robustness but can be slow, unavailable, privacy-sensitive or temporally unstable. A detector for live traffic should report performance with realistic feature outages and on the naturally skewed base rate of harmful URLs, not just on a balanced test table.[1]
Federated learning does not by itself provide privacy
Keeping raw URLs local can reduce one route of exposure, but the experiment does not define an adversary such as an honest-but-curious server, colluding clients or an external eavesdropper. It does not add secure aggregation, differential privacy, update clipping or defences against malicious participants. Model updates can leak information, and a client can poison a shared model. The paper's result is therefore about distributed optimisation and accuracy, not proof that participant data are confidential or the system is secure.
The dataset itself is a fixed feature table, so both branches of the hybrid derive from the same 87 engineered inputs; no raw URL-string or web-page modality is tested. All 20 clients come from partitioning one central benchmark rather than separate organisations with different collection tools, jurisdictions and attack periods. The authors received no external or internal funding and declared no competing interests. Their data and code-availability statement points to the public benchmark and an archived implementation, supporting replication, but current phishing drift remains outside the design.[1]
What would change the assessment
The decisive next step is a prospective, time-separated federation across real organisations or independently collected datasets. Each client should retain genuinely local traffic, report its collection period and language, and test a frozen model on later campaigns and unseen domains. The evaluation should include naturally imbalanced data, feature outages, adversarial manipulation, calibration, false-positive workload and performance for the weakest participant. A secure design should state its threat model and test secure aggregation, differential privacy and poisoning defences rather than treating local storage as a complete privacy claim.
The present paper supports two bounded conclusions. First, careful paired testing confirms that XGBoost is a strong and fast central baseline on this engineered benchmark. Second, federated SAFW-Hybrid and purpose-built federated boosting can approach that central performance when one dataset is divided among 20 simulated clients. It does not show that the approach blocks phishing in browsers, stays accurate as campaigns change, protects updates from inference or sabotage, or works across independent organisations. Those are the results defenders need before the simulation becomes operational evidence.[1][2]
What this means for people
- Distributed training could let organisations learn from broader attack patterns without centralising every raw URL record.
- False positives can block legitimate sites and create investigation workload; the paper does not measure either consequence.
- People and organisations should not treat federated optimisation alone as proof that sensitive telemetry or model updates are private.
Global context
The authors are based in Indian computer-science and cyber-forensics institutions, while the benchmark is a public international research dataset. Cross-border threat sharing faces legal, linguistic and infrastructure differences that a simulated partition cannot reproduce. The study is relevant to organisations exploring collaborative detection, but local data governance, threat models and independent temporal validation remain prerequisites.
What the evidence does not yet show
- All examples come from one balanced public benchmark with engineered features rather than current, independently collected organisational traffic.
- The 20 clients and their non-IID distributions were simulated by partitioning one central training set.
- The study provides no formal privacy guarantee, secure aggregation, differential privacy, poisoning defence or explicit adversary model.
- The fastest latency assumes externally retrieved google_index and page_rank features are cached; cold lookups add an estimated 100–500 ms.
- A global federated score masked much weaker local-client accuracy in some runs.
- No prospective time split, live browser deployment, user-impact study or evaluation against adaptive attackers was conducted.
What to watch next
- Time-separated tests on later phishing campaigns and independent multilingual datasets.
- A real cross-organisation trial with per-client accuracy, calibration and false-positive workload.
- A stated threat model plus secure aggregation, differential privacy and poisoning-resilience evidence.
- End-to-end latency and robustness when external reputation features are unavailable or stale.
Living evidence record
Impact record IAI-0XPH16I
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Present
Record status
Monitoring
Last checked
5 October 2026
Source trail
2 direct sources across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Related-source reporting disclosure
This record analyses 2 linked source records around the same underlying development. The extra records add method, date or context, but they do not by themselves constitute independent replication of every performance claim or predicted outcome.
Evidence trail
Sources used for this report
Links checked 5 October 2026
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
Continue the story
Related reporting
Security & Defence
Can clouds share phishing signals without sharing raw data?
A three-model ensemble reached 95.21% accuracy on 2,211 test examples while exchanging probability scores. But the clouds, network and feature split were simulated, and the framework has no formal privacy or adversarial-robustness guarantee.
8 min · 1 source
Security & Defence
Can page content expose malicious websites?
A peer-reviewed benchmark of 689,556 webpages finds that analysing visible page content alongside the URL sharply reduced errors compared with URL-only models. The random stratified split, previously detected threats and missing per-page language labels leave live-world drift and novel attacks unresolved.
7 min · 1 source
Security & Defence
Can AI detect attack types it never saw in training?
A hybrid Transformer–LSTM detected two held-out categories in the UNSW-NB15 benchmark, with 88.13% recall on the combined unseen subset. It was a binary, row-level laboratory test—not evidence of live zero-day defence.
8 min · 1 source
Reader commentary
Add evidence, experience or a question
No account is required. Reader notes are published after a brief civility, relevance and safety check; disagreement is welcome.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.