Skip to content
XSCRIPTOR/DEV~$ _
~/resources/web/xwa/shinobi/
├──A stealth web scraper with a single web interface. Downloads entire sites — HTML, CSS, JS, images, PDFs — while evading detection through multiple anti-blocking layers: user-agent rotation, header randomization, request delay with jitter, exponential backoff, and proxy support.
├──Anti-BlockingUA rotation, header randomize
├──Deep CrawlBFS recursive with depth
├──JS RenderingHeadless Chromium
├──Asset DownloadHTML/CSS/JS/images/PDF
├──Proxy RotationHTTP/HTTPS/SOCKS5
└──Fast ModePure Rust — zero deps