Statutory Document Monitoring System Development

Item 1 of 3

Technologies:

Docker (9)
PostgreSQL (27)
Auto Testing (25)
Django (23)
Linux (23)

Domains:

Business Solutions (12)
Finance and Cryptocurrency (7)

Project Goals

The goal of the Statutory Document Monitoring System project, named StatStat Monitor, was to develop a tool capable of detecting changes in the statutory documents of specified enterprises. The system aimed to provide early alerts to security services and relevant stakeholders, allowing them to take proactive measures in response to significant changes that could indicate risks such as bankruptcy, fraud, or power shifts.

Functional Capabilities

  • Data Parsing from Tax Office Website: The system accessed the official tax office website (nalog.ru) to download statutory document data for specified enterprises. It then parsed the data and identified differences in document versions, focusing on critical changes that could impact the enterprise's stability.
  • Change Detection and Notification: The system monitored the OGRN (Principal State Registration Number) and other statutory documents for changes, such as modifications in ownership or structural details. Any detected changes were flagged, and notifications were sent to the security service for prompt action.
  • Captcha-Solving Implementation: To access data from nalog.ru, the system had to bypass captcha challenges. A captcha-solving server was implemented to automate this process, ensuring uninterrupted data retrieval.
  • Proxy Server Sets for Parsing: To avoid being blocked by the tax office website during frequent data requests, the system used proxy server sets for parsing. This allowed for reliable data extraction without interruptions.
  • Administrative Interface: A minimal Python Django interface was developed for administrative purposes, allowing operators to manage the list of monitored enterprises and set up recipients for document change notifications.
  • Email Notifications: The system's user interface took the form of email notifications, providing relevant stakeholders with timely information about changes detected in statutory documents.

Solution Concept

The Statutory Document Monitoring System was designed to address a critical challenge faced by enterprises and security services: tracking significant changes in statutory documents that may indicate potential risks, such as bankruptcy, fraudulent schemes, or power redistribution. Given the large volume of data and the frequency of changes, manually monitoring these documents was impractical.

The system accessed the nalog.ru website to download data related to statutory documents, including OGRN and ownership details. It then compared current and previous versions of the documents to identify any changes. This information was processed and communicated to security services via email notifications.

The backend was developed using Python and Django, while Docker was used to ensure consistent deployment. The system employed captcha-solving and proxy server solutions to facilitate data retrieval from the tax office website. The focus was on creating a robust and reliable tool that could handle a large number of enterprises simultaneously, ensuring that security services were always informed about critical changes in statutory documents.

Results

  • Facilitated Tax Services Operations: The system significantly facilitated the work of tax services by simultaneously covering a large number of enterprises, ensuring that critical changes in statutory documents were detected and reported.
  • Reduction in Fraudulent Activities: By providing early alerts about significant changes in statutory documents, the system helped reduce the number of fraudulent schemes and suspicious activities within the monitored enterprises.
  • Increased Speed of Detection: The system enabled rapid detection of changes, allowing security services to respond promptly to potential risks, such as fraudulent schemes, bankruptcy, or power shifts.

Technologies and Architecture

  • Backend Development:
    • Python and Django: Used for developing the core application and administrative interface, providing a scalable and robust solution for managing statutory document monitoring.
  • Data Parsing and Retrieval:
    • Data Parsing from nalog.ru: The system accessed the official tax office website to retrieve statutory document data, including OGRN and ownership details.
    • Captcha-Solving Server: Implemented to bypass captcha challenges on nalog.ru, ensuring consistent and reliable data retrieval.
    • Proxy Servers: Used to avoid being blocked by the tax office website, allowing the system to perform frequent data requests without interruption.
  • Database Management:
    • PostgreSQL: Utilized as the primary database for storing document versions, monitoring changes, and managing notifications.
  • Deployment and Integration:
    • Docker: Used for containerizing the application, ensuring consistent deployment across different environments.
  • Operating Systems:
    • Linux: Supported as the server-side environment, providing flexibility and scalability for the backend components.

User Cases

  • Security Service Personnel: Security team members used the system to receive timely alerts about changes in statutory documents, allowing them to take proactive measures in response to potential risks.
  • Tax Services: The system facilitated the work of tax services by automatically monitoring statutory document changes for a large number of enterprises, reducing the need for manual tracking.
  • Enterprise Stakeholders: Relevant stakeholders received email notifications about significant changes in statutory documents, enabling them to stay informed and make decisions accordingly.

Integration and Development Process

  • Requirements Gathering: The project began with gathering requirements from the enterprise's security and tax services teams to understand their specific needs for monitoring statutory documents and identifying potential risks.
  • System Design and Architecture: The system architecture was designed to support the automated retrieval and analysis of statutory documents. The backend was developed using Python and Django, while Docker was used for containerization to ensure consistent deployment.
  • Team Formation and Leadership: A team of software developers, system architects, and data analysts was formed to develop and implement the system. The development process followed the Agile Scrum methodology, allowing for continuous feedback and iterative improvements.
  • Implementation and Testing: The system was implemented iteratively, with regular testing to ensure that the document monitoring capabilities met the needs of the security and tax services teams. The use of captcha-solving and proxy servers ensured reliable data retrieval from the tax office website.

Client Benefits

  • Enhanced Monitoring Capabilities: The system provided the enterprise with a powerful tool for monitoring statutory document changes, ensuring that critical changes were detected and reported in a timely manner.
  • Reduced Fraud and Manipulation: By identifying significant changes in statutory documents, the system helped reduce fraudulent activities and manipulations, leading to increased security and stability for the enterprise.
  • Improved Efficiency: The automation of document monitoring reduced the time and effort required for tax services to track changes manually, allowing them to focus on addressing potential risks rather than searching for them.
  • Proactive Risk Management: The system enabled security services to take a proactive approach to risk management by identifying potential risks before they could escalate, reducing the likelihood of fraud, bankruptcy, and other incidents.