Ever since I first publication "Burstiness and Memory successful Complex Systems" by Kwang-Il Goh, I person been obsessed pinch the 2 formulas showcased successful the paper. Burstiness (B), and its just-as-important counterpart Memory (M), fto america understand the dynamics of event-based systems.
We tin usage them to analyse the behaviour of sent emails, texts aliases moreover heartbeats erstwhile only the clip astatine which those events happened is known. This allows america to intelligibly found which patterns are human-like and which aren't based connected a dataset of already classified data. We cognize humans reply to texts successful a bursty mode (the clip they return to reply is not uniform) while elemental bots respond arsenic accelerated arsenic possible, frankincense they person different B and M coefficients.
Goh, K.-I., & Barabási, A.-L. (2008), Figure 4
In erstwhile individual research, I utilized these 2 values to differentiate betwixt quality and bot sessions based connected the timings of their requests, which usually worked, nonetheless, this attack identified sessions arsenic quality if they were sent from immoderate browser that loaded CSS and JS files, allowing much precocious bots for illustration ClaudeBot to look quality by utilizing a headless browser. It fundamentally distinguished betwixt crafted requests versus requests sent from a existent browser, alternatively of bot and human.
The scroll wheel
Some clip agone I noticed a distinctly quality behaviour that I judge cannot beryllium easy imitated by bots, scrolling. When I americium scrolling a page looking for accusation utilizing the scroll wheel, I usually don't scroll down linearly until I find what I americium looking for, instead, I do truthful pinch a bursty pattern. For example, that shape mightiness much intimately lucifer clip bid a aliases e alternatively than c.
Goh, K.-I., & Barabási, A.-L. (2008), Figure 1
I deliberation astir scraping bot developers still haven't honed their scrolling pattern, truthful I decided to trial if burstiness and representation applied to the inter-event times of scroll events were variables that could person predictive powerfulness successful a instrumentality learning model, successful bid to separate betwixt humans and bots.
To do so, I utilized the dataset provided by the insubstantial "FP-Agent: Fingerprinting AI Browsing Agents" by Ethan Wang, et al., successful which the authors explained the process of really they created a instrumentality learning exemplary that could separate betwixt different AI browsing agents successful a controlled situation (a website made specifically for the purpose). They recorded information of different agents and humans navigating done their website. Their exemplary gave bully results, but I don't deliberation it would beryllium effective successful a real-world mounting erstwhile applied to different websites, which involves tons of variation. Nevertheless, they made the dataset disposable for download connected OSF.io.
I calculated the burstiness and representation values for the JavaScript "scroll" events of each supplier successful each page (one burstiness and representation worth per page visited). These are the results:
We tin observe that the quality distribution is intelligibly different from the different agents', presenting a higher Burstiness coefficient and almost nary Memory; the only supplier that sometimes resembles humans is ChatGPT Agent. This information seems promising, but to further beryllium if the 2 values person predictive worth I trained a LightGBM (decision character gradient boosting) exemplary to categorize each single-page interaction.
The exemplary only had 2 features available, B and M, and it had to foretell which supplier performed the interactions. The consequence was an accuracy of 73.4%, which is not that bad. As expected, the model mainly confused Humans and ChatGPT Agent, astir apt owed to debased characteristic count aliases a mini dataset (~150 information points per supplier is excessively small), the exemplary would request different features to beryllium capable to separate them further.
Conclusion
Scroll behaviour seems to beryllium a applicable information constituent for distinguishing bots from humans, and Burstiness and Memory person proven to person predictive powerfulness successful a controlled environment. This attack will evidently not beryllium effective connected websites that don't require scrolling; connected the different hand, blogging aliases accusation sites for illustration Wikipedia are the ones astir capable to return advantage of this method.
The exemplary shown successful this blog station is not meticulous enough, though, if mixed pinch much features extracted from different actions specified arsenic rodent activity aliases typing, I judge it is imaginable to create a wide exemplary tin of protecting websites from scraper bots.
I will support posting my investigation successful this website. Next up, I will beryllium taking a look astatine rodent activity patterns.
English (US) · 
Indonesian (ID) · 
