Auto-Mobile Data-Driven System

The data-driven system is a specialized software solution for information collecting, administration, and presentation.
Auto-Mobile Data-Driven System
The Auto-Mobile Data-Driven System is a technology that collects current and trustworthy auto-mobile data from 30k American auto-mobile websites.
Modules
There are 5 modules in the Auto-Mobile Data-Driven system.
1st Module: In the first module, 30k websites in the US are crawled for auto-mobile data (VIN, Year, Model, Trim, Make, Mileage, Dealer Information, and so on).
2nd Module: The second module is the data management module. MongoDB is used to store the scraped data, and it is refreshed every day.
To organize and handle data, the database comprises 10 connected tables. The database is routinely updated with current data and is free of duplicate records.
3rd Module: This module is for data maintenance. In order to keep the data current, 10 VMs are used to deploy the scrapers. Every day, the scrapers are started by the cronjob setting.
4th Module: This is the Data Analysis Module. The purpose of the data analysis module is to predict future changes and extract useful information from the raw data, after which the database will be updated with the analyzed data.
5th Module: This is the Data Visualization Module. The purpose of this module is to display database information and simplify data management for users.
Users can trigger dashboard scrapers selectively by being assigned specific roles by the admins, which makes it easier for users to obtain data. Additionally, they can monitor both raw and processed data.