Commit Graph

5 Commits

Author SHA1 Message Date
Louis Mylle
c6a69c6ada feat: Add Bookys scraper alongside EBoek with startup mode chooser
Adds a second scraper for bookys-ebooks.com that harvests direct 1fichier
download URLs into a .txt file. The existing EBoek scraper core is unchanged;
only the GUI gained a Mode menu entry to switch between the two.

Bookys specifics:
- Visible (never headless) browser, since the site is behind Cloudflare.
  The challenge is auto-detected: the scraper polls for real listing items
  and continues on its own once it clears, no button needed.
- Persistent Chrome profile in ~/.eboek_scraper/bookys_chrome_profile so the
  Cloudflare clearance cookie survives between runs. Removed the hardcoded
  user-agent override, which claimed Chrome/120 against a real Chrome/150 and
  invalidated that cookie on every launch.
- Pop-up ads are avoided by never clicking host links: hrefs are read and
  navigated to directly, with window.open neutered via CDP as a backstop.
- Host links are read with textContent rather than Selenium's .text, which
  returns empty for these elements because they sit in a collapsed container.
- Only 1fichier hosts are followed; other hosts on a page are ignored.

Build configs list the new modules as hidden imports in all four places.
gui_main imports the Bookys window lazily, so PyInstaller's static analysis
would otherwise miss it and the .exe would crash on switching modes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NVJc2XuTvqBHSuqotetseS
2026-07-19 20:05:49 +02:00
Louis Mylle
4b65cbbe84 Merge branch 'main' of https://git.xbastards.com/LouisMylle/eboek.info-scraper 2026-01-10 17:36:26 +01:00
Louis Mylle
610a20d12d feat: Enhance scraper timing configuration with user-defined settings and UI adjustments 2026-01-10 17:36:09 +01:00
louis
cf9ce084a5 Fix windows 2026-01-10 15:44:14 +01:00
Louis Mylle
ea4cab15c3 feat: Add installation scripts for Windows and Unix-based systems
- Created `install_and_run.bat` for Windows installation and setup.
- Created `install_and_run.sh` for Unix-based systems installation and setup.
- Removed `main.py` as it is no longer needed.
- Updated `requirements.txt` to specify package versions and added PyQt5.
- Deleted `start.bat` as it is redundant.
- Added unit tests for core functionality and scraping modes.
- Implemented input validation utilities in `utils/validators.py`.
- Added support for dual scraping modes in the scraper.
2026-01-10 14:45:00 +01:00