Skip to content

Releases: S4M8/intel-extractor

Intel Extractor v0.2.3

02 Jul 05:05
ee7a54c
Compare
Choose a tag to compare

Intel Extractor: v0.2.3 - Major Scraping Improvements and Database Integration

0.2.3 Release Notes:

  • Efficient Scraping: The scraping strategy has been revamped to directly target the Star Citizen API, eliminating the need to scroll through the DOM. This translates to faster and more reliable data extraction.
  • Organization Verification: The tool now includes functions to verify the existence of an organization before proceeding with data collection.
  • Pagination Handling: Intel Extractor can now determine the number of pages required to scrape all member dossiers within an organization, ensuring comprehensive data capture.
  • Detailed Citizen Data: Extracted information now includes not just basic details like name and handle, but also location and fluency information from individual member profiles (optional).
  • SQLite Database Integration: Scraped data is now stored in a centralized SQLite database, facilitating easier management and analysis.
  • Separate CSV Export: While data is stored in the database, a separate CSV file containing all scanned member URLs is still generated for reference.

What's Changed

  • Electron wrapper by @neon-nomad in #1
  • org-scan by @neon-nomad in #2
  • Add Existing Roster Input by @neon-nomad in #3
  • Scraping improvements by @neon-nomad in #4

Full Changelog: https://github.com/neon-nomad/intel-extractor/commits/v0.2.3