Archive tester

Facebook: https://www.facebook.com/archive.tester/
If you find this software useful, please consider making a donation to support development.

Description

"Archive tester" is a tool to facilitate testing integrity of massive collections of mixed archive files such as *.7z, *.bz2, *.gz , *.rar *.tar, *.xz, *.zip, *.exe, *.r00/r01, *.z00/z01, *.001... and *.flac!

How it helps

"Archive tester" is a UI relying on UnRAR.exe (included), 7-zip (can be configured) and or flac.exe (can be configured).

It helps by

  • Scanning your drives or folders for testable files, based on file extension
  • Queuing the testing of mixed archives types (no need to launch tests separately and/or go from one software to another)
  • Running parallel tests (~multithreading), which speed testing of multiple single-threaded algorithm encoded files
  • Generating logs for each tested files (or optionally only for unsuccessful tests).
  • Settings can help tuning and scaling the work.

"Archive tester" is here to bring you bad news: some of your archive files are (probably) broken, corrupted... and if they don't include recovery data and if you don't have backups, they may be lost forever.
Well, hopefully this is not your story! "Archive tester" is a tool to help detecting corrupted archive files across multiple paths/drives showing progress indicators and writing logs. Just give it a few settings and let it run.

Since the end of the nineties, yes that was last century, I have been into various kinds of digital creations. These last years I started doing photography and even more video than I did in the past. Hence I accumulated approximately 25 to 30 terabytes of data.

Over time, I lost a very few archives... Yet some were corrupted but I managed to recover most of those. I have in mind to, someday, write a more detailed page about my archiving processes. Not to tell you what you "should" do but in case you find some inspiration there.

Someday, I tested the integrity of a hard disk using HD Tune. The test passed. Anyway, immediately after, I tested the same disk using the constructor's testing tool, but then, the test failed! All other similar disks from the same brand passed tests, so this was not a constructor conspiracy to make me buy another hard disk. (I think it was a Western Digital Green or a Samsung Spinpoint, I had a lot of both.)
First lesson, all disk testing softwares are not made the same. (But this is not the issue that is address by "Archive tester".)
Yet, on a some drives that passed all tests, I still happened to find some corrupted archives.
Second lesson, testing disk surface integrity is a thing, but you really need to test integrity at file level as well.

Since then, I put everything I archive in *.rar archives, even when there is no actual space gain because this way they are testable and, to some extent, recoverable *. Yet this testability can be automated!

I know that there are command lines that allow to test paths recursively, but for long and numerous archive tests I wanted something more handy that would give me more control, more progression information and would already do a part of the work of reading the logs for me. That's why I started "Archive tester". **

* RAR's recovery record feature saved multiple files for me already. To achieve the same with other formats that I know, you have to generate extra parity using PAR2.

** Command lines also happen to do redundant testings of multivolume archives. I didn't investigate this that much.

Downloads

Link Version Date Size SHA256 hash Requires uninstalling previous version
TC-Archive_tester-0.2.0.0-Setup.7z
TC-Archive_tester-0.2.0.0-Setup.rar
TC-Archive_tester-0.2.0.0-Setup.zip
0.2.0.0 2018-07-18 1,09 MB (1.146.433 bytes)
1,13 MB (1.186.162 bytes)
1,09 MB (1.148.634 bytes)
62D96DE1685F86C35670D74C5A5FE67983D0011B5E2D994CE50606FA4B06B200
E4AD7A5BCAC93CDED2661C6192D688CB7EB7B312B89D6A89495FF71B4E80792E
A99E72C393AF34563551FD3CCBA274655A59B5F1983D8DCAF5BD63DEA74DE8ED
No
TC-Archive_tester-0.1.4.1-Setup.rar 0.1.4.1 2018-01-14 1,12 MB (1.185.749 bytes) D812DFE077E268A410DE4CDEC23FB45EFF7C3C86605BA91BDDE66F82CC1B3D36 Recommended!
Hopefully for the last time as installer software changed.
Link Version Date Size SHA256 hash Requires uninstalling previous version
TC-Archive_tester-0.1.4.0-Setup.rar 0.1.4.0 2017-09-28 819 KB (839.208 bytes) 16C09B2CEF1E62B4B30DB3F205E2003F475DF50E1962EEE785F682F590D4E7D3 No
TC-Archive_tester-0.1.3.2-Setup.rar 0.1.3.2 2017-09-21 815 KB (835.373 bytes) 38E0106AADF15497D55EB6E64D62C333E9B482A2083837DEA5234C460AB82ACC Yes
TC-Archive_tester-0.1.3.1-Setup.rar 0.1.3.1 2017-09-19 820 KB (839.680 bytes) 2068B21BED99B9224D3E4C0A1ECE69CF6D6EEF82F8ED46B489E1642D091CB368 Yes
TC-Archive_tester-0.1.3.0-Setup.rar 0.1.3.0 2017-09-19 815 KB (835.058 bytes) 5647F8D82B59092AE9A1CAF2056A949C1550AEDDC089964CC2716EDA2BFE713A No
TC-Archive_tester-0.1.2.0-Setup.rar 0.1.2.0 2017-09-10 375 KB (384.330 bytes) CB5726D4C46DDA8F888D453BE0CE5E3597821C9FCAC3782F3F1E7DEB2A6BAEDE No

Screenshot

TC's Archive tester v0.1.3.0 screenshot

Setup

  1. Download the software package from this page, extract the files and run "setup.exe".
  2. At first run, you have to agree with the licence.
  3. Go to "Settings" tab.
    1. In "Preferences" subtab, the path to the folder where you want logs to be written in "Log folder" field. This is where the result logs will be generated. If it does not exist yet, it will created when the first log file need to be written.
    2. In ".exe files" subtab, fill the paths where "7z.exe" and "flac.exe" can be found. UnRAR.exe should be set already as its license allows its inclusion to this software.
      (You may find 7z.exe through http://www.7-zip.org/ and flac.exe through https://xiph.org/flac/ )

There are some more settings in the "Settings" tab. For now I'll let you explore them.

Usage

For now, you can find some more explainations in the "About" tab.
  • Windows running .NET framework 4.0+
  • 7z.exe (tested with version 16.04 [64] ) to test *.7z, *.bz2 (BZip2), *.gz, *.tar, *.wim, *.xz, *.zip files. Output needs to be in English!
  • flac.exe (tested with version 1.3.2 win32/win64 tested) to test *.7z files. Output needs to be in English!
  • To test *.rar files, UnRAR.exe is provided with the application. Output needs to remain in English!

Version 0.2.0.0 - 20180718

  • Parallel testing (~multithreading)
    • Testing processes can now be ran in parallel (multithreaded). Default thread count is based on .NET's "Environment.ProcessorCount" which counts your computer's logical cores. There is a setting to change this manually too.
      Previous version was singlethreaded, relying on the fact the algorithms such as RAR5 are multithreaded. But some, for instance RAR4, are not multithreaded. This should bring a huge performance gain when testing a large number of files encoded with singlethreaded algorithms (or at least when testing algorithms are singlethreaded).
  • Multivolume: I still have a more flexible file extension/tester matching system in mind, but here is already something that should solve many cases.
    • *.001 are now testable through 7z.exe
    • *.r00 and *.r01 are now testable through UnRAR.exe, *.r01 is considered non testable if matching *.z00 exists in the same folder.
    • *.z00 and *.z01 are now testable through 7z.exe, *.z01 is considered non testable if matching *.z00 exists in the same folder.
    • Prevented case when all files ending with *.rar but belonging to the same multivolume archive would get tested along with each other. Which resulted in n2-1 redundant tests (if I got the formula right). Files ending with ".part1.rar" are still considered testable while all other volume, following pattern ".part[N].rar" are now considered non-testable.
    • ?: Should multivolume non-first volumes be counted as non-testable or simply ignored? For now they are counted as non-testable.
  • Self-extracting I still have a more flexible file extension/tester matching system in mind, but here is already something that should solve many cases.
    • *.exe are now optionally testable. *.exe first get tested with UnRAR.exe, if not a RAR archive, a *.exe file will then be tested by 7z.exe, if failed, it will be counted as failed test. There is a checkbox to test or ignore *.exe files. Default value: false.
  • Benchmarking:
    • Added "kB/s" indicator=
  • Targeting tuning:
    • In the "Inputs: Paths to be tested" field, you may now include/exclude specific files, the same way you would include/exclude a folder.
    • There is now a setting to automatically skip some folders by name (separated with "|"). Default value: System Volume Information | $RECYCLE.BIN. (In previous versions, "$RECYCLE.BIN" was hard-coded.)
    • Prevented cases when some file could be tester multiple times or counted multiple times as non testable files (typically if a folder and its parent folder were both set as inputs).
  • Stability and efficiency
    • Closing main form/application now more efficiently closes background threads.
    • Stopping during discovery process will now be much faster (kind of instant).
  • UI:
    • At first start, the input text field is now filled with a demo/tutorial, that should be able to scan you all computer if started.
    • Tests that have been stopped by clicking on "Stop" now count as skipped.
    • Possible future improvement: Progress indicators work fine is the full process is ran. Using "Skip" or "Stop" buttons result in skipping files, which may make progress indicators interpretation arguable. This does not look like a priority but may possibly be re-thought in the future. Probably in a smaller update.
  • UI consistency:
    • "Skip" button is now disabled during the discovery process and only activated during testing process.
    • Progress indicators that can only be evaluated after at least one file has been tested (such as "Est. total time") now read "?" before they can be evaluated, instead of "00:00:00.0000000".
    • UI consistency: "Launch/Testing" button now reads "Scanning" during the scanning process (instead of "Testing").
    • "Time spent on current file" is replaced with the count of active parallel tests when parallel testing.
    • .exe path text boxes and most setting buttons are now disabled while processing.
  • Dependencies and compatibility:
    • Tested with UnRAR.exe 5.60 and 7-Zip 18.05.
    • Update UnRAR.exe to version 5.60
  • Possible issue(s):
    • On my Win7 computer, stopping the testing process ("Stop" button") gets the application stuck with one running parallel test running (according to UI). This did not happen on the Win10 computer where I write the code. Cause unidentified so far.

Version 0.1.4.1 - 20180114

  • Fixed a bug which could possibly cause false positives or false negatives in rare cases where specific words appeared in tested file names or paths.
  • Fixed progress bar not being reset when launching tests. (It was re-synching after first file was tested, which made this bug unnoticeable when testing small files.)
  • Fixed a multithreading "bug" which could make some labels not update their value on some systems. However this slightly slows the UI, especially at discovery time, but I think that information accuracy is more important.
  • UI elements such as text fields, checkboxes, radio buttons etc. are now disabled during process.
  • If some tester .exe path is filled in "Configuration" but actual tester .exe file is not found, associated files now go to a new "Tester_not_found" output channel/folder.
  • If some tester .exe path is not filled in "Configuration", files of associated types now go as "Non testable files".
  • Forced 7-Zip output to be UTF-8.
  • Using "Inno Setup" installer instead of "Microsoft Visual Studio 2017 Installer Projects". "Inno Setup" offers more flexibility regarding version numbers, licence agreement, uninstaller, icons, ...
    • In Windows "App & features", version numbers are now shown as x.x.x.x instead of x.x.x.
    • As it is now included in installer, licence agreement as been deactived in the software.
    • "Archive tester" as a new icon. As well as a new uninstaller icon for Start Menu.
    • Hopefully it will not be necessary to uninstall previous versions when installing some new versions in the future.
  • Prevented a rare bug that could happen when using Stop or Skip button, caused by trying to check the state of a process that meanwhile had already finished (only seen in debug mode).
  • Slightly optimised the way the tester .exe files paths are tested if a valid one is not provided already (especially regarding "UnRAR.exe" and secondly it will also look for "WinRAR.exe" standard installation path if "UnRAR.exe" was not found).
    • A temporary feature allows to ignore install version of some type of tester by writing "Do not test" in the associated configuration field, resulting in associated files being "Non testable". (This is temporary because a more generic way of associating files and testers is planned in future versions.)
  • Application setting "Seven7zExePath" renamed to "SevenzExePath". Because that's how I write in the rest of the code.

Version 0.1.4.0 - 2017-09-28

  • Skip button and timeout in "Configuration" > "Settings". Because some large corrupted archives files may take very long, possibly cause a background timeout. Skipped files get removed from progress counts.
  • Ability to exclude paths: Add a "-" (minus) at start of the line to exclude a folder.
  • Progress is now bar based on byte progress instead of file progress.
  • "Settings" tag renamed to "Configuration", new "Settings" tab added under "Configuration" tab.

Version 0.1.3.2 - 2017-09-21

  • Fixed: Bug that could stop file discovery in folder containing non-testable files.

Version 0.1.3.1 - 2017-09-19

  • Fixed a bug at application start due to name change of the licence file.
  • Fixed: Hopefully fixed a possible crash on application close.
  • Fixed: Time spent on current file time label which was not exactly behaving as the other time labels.

Version 0.1.3.0 - 2017-09-19

  • More archive types tested through 7z.exe: *.bz2 (BZip2), *.gz, *.tar, *.wim, *.xz
  • FLAC: *.flac files may now be tested through flac.exe. (You will need to configure flac.exe path in the "Settings" tab.)
  • "Archive tester" is now using UnRAR.exe instead of RAR.exe. UnRAR.exe is included in "Archive tester" package as its license allows it. You may still point to another *.exe file if you wish.
  • In "Settings" tab, you now point at *.exe files instead of a folder + hardcoded *.exe file.
  • Stop button can now stop process while a file is being tested.
  • New "Time spent on current file" timer.
  • Two new log "channels": "Exceptions at discover time" and "Exceptions at testing time". Hence, the exception display form is less often used.
  • Most Exceptions should not stop the process anymore.
  • Application name set to "Archive tester", and "ArchiveTester" as machine friendly name. Exit "TC's Archive Tester" etc.
  • Estimated remaining time label is not shifted to the left anymore.
  • Progress labels now have colors: Green for success, orange for things that may be acceptable but you should give a check, red for failed things.
  • Refactoring for features "extensibility".
  • Application and shortcuts icons should now be displayed correctly.
  • Fixed: UTC checkbox had no effect.
  • License added.
  • ...

Version 0.1.2.0 - 2017-09-10

  • First release on website
  • Fallback log: A secondary window used if, for any reason, log files could not but written (e. g., too long path for Windows)
  • Exception display window: A secondary window that can display possible exceptions/errors
  • ...
  • Process could get broken if .exe files command line outputs change. In that case you would most likely get "Failed" test results for files that are not actually broken. (Which is safer than the other way around.)
  • As 99.99% of my archives are *.rar files, and even my *.flac files are in *.rar archives, RAR is the format I tested the most by simply using it.

As for many things, I see A LOT of ways I can improve this application. Progression will depend on the time I find for it. Obviously, in case it meets success and generates some money through donations, I may give it more attention.

Features and improvements ideas

  • Drag and drop to "Inputs" field
  • Making treatment by file type more flexible
  • Handling multi-volumes
  • Check if there is a chance that redundant paths get tested multiple times and prevent this to happen
  • What to do with auto-extract *.exe archives?
  • Replacing the licence screen with "Welcome" screen including the licence and some required settings such as .exe file paths
  • Make logs that arenicer to read? HTML? XML/XSLT? JSON/JS?
  • ...

Coding and debugging ideas

  • Use DLLs instead of command line? Do such DLLs able of testing files actually exist? For .NET?
  • For now, it is often necessary to uninstall previous version before installing new version. This is related to version numbers.
  • ...

Support ideas

  • Video tutorial(s)?
  • Adding some screenshots here
  • ...