FastFileCheck work
an invention by pstivanin
Updated 5 months ago. No love. 1 follower.

Description

FastFileCheck is a high-performance, multithreaded file integrity checker for Linux. Designed for speed and efficiency, it utilizes parallel processing and a lightweight database to quickly hash and verify large volumes of files, ensuring their integrity over time.

https://github.com/paolostivanin/FastFileCheck

Goals

  • Release v1.0.0

Design overwiew:

  • Main thread (producer): traverses directories and feeds the queue (one thread is more than enough for most use cases)
  • Dedicated consumer thread: manages queue and distributes work to threadpool
  • Worker threads: compute hashes in parallel

This separation of concerns is efficient because:

  • Directory traversal is I/O bound and works well in a single thread
  • Queue management is centralized, preventing race conditions
  • Hash computation is CPU-intensive and properly parallelized

Looking for hackers with the skills:

c hashing lmdb multithreading integrity-checker

This project is part of:

Hack Week 24

Activity

  • 5 months ago: pstivanin added keyword "c" to this project.
  • 5 months ago: pstivanin added keyword "hashing" to this project.
  • 5 months ago: pstivanin added keyword "lmdb" to this project.
  • 5 months ago: pstivanin added keyword "multithreading" to this project.
  • 5 months ago: pstivanin added keyword "integrity-checker" to this project.
  • 5 months ago: pstivanin started this project.
  • 6 months ago: pstivanin originated this project.


    • Comments

    • pstivanin
      5 months ago by pstivanin | Reply

      most of the stuff have been implemented: https://github.com/paolostivanin/FastFileCheck/releases/tag/v1.0.0-alpha1

    • pstivanin
      5 months ago by pstivanin | Reply

      deleted

    Similar Projects

    This project is one of its kind!