Archive tester

Facebook: https://www.facebook.com/archive.tester/
If you find this software useful, please consider making a donation to support development.

Description

"Archive tester" is a tool to facilitate mass testing of collection of mixed archive files such as *.7z, *.bz2, *.gz , *.rar *.tar, *.xz, *.zip... and *.flac!

"Archive tester" is here to bring you bad news: some of your archive files are (probably) broken, corrupted... and if they don't include recovery data and if you don't have backups, they may be lost forever.
Well, hopefully this is not your story! "Archive tester" is a tool to help detecting corrupted archive files across multiple paths/drives showing progress indicators and writing logs. Just give it a few settings and let it run.

Since the end of the nineties, yes that was last century, I have been into various kinds of digital creations. These last years I started doing photography and even more video than I did in the past. Hence I accumulated approximately 25 to 30 terabytes of data.

Over time, I lost a very few archives... Yet some were corrupted but I managed to recover most of those. I have in mind to, someday, write a more detailed page about my archiving processes. Not to tell you what you "should" do but in case you find some inspiration there.

Someday, I tested the integrity of a hard disk using HD Tune. The test passed. Anyway, immediately after, I tested the same disk using the constructor's testing tool, but then, the test failed! All other similar disks from the same brand passed tests, so this was not a constructor conspiracy to make me buy another hard disk. (I think it was a Western Digital Green or a Samsung Spinpoint, I had a lot of both.)
First lesson, all disk testing softwares are not made the same. (But this is not the issue that is address by "Archive tester".)
Yet, on a some drives that passed all tests, I still happened to find some corrupted archives.
Second lesson, testing disk surface integrity is a thing, but you really need to test integrity at file level as well.

Since then, I put everything I archive in *.rar archives, even when there is no actual space gain because this way they are testable and, to some extent, recoverable *. Yet this testability can be automated!

I know that there are command lines that allow to test paths recursively, but for long and numerous archive tests I wanted something more handy that would give me more control, more progression information and would already do a part of the work of reading the logs for me. That's why I started "Archive tester". **

* RAR's recovery record feature saved multiple files for me already. To achieve the same with other formats that I know, you have to generate extra parity using PAR2.

** Command lines also happen to do redundant testings of multivolume archives. I didn't investigate this that much.

Screenshot

TC's Archive tester v0.1.3.0 screenshot

Downloads

Link Version Date Size SHA256 hash Requires uninstalling previous version
TC-Archive_tester-0.1.4.1-Setup.rar 0.1.4.1 2018-01-14 1,12 MB (1.185.749 bytes) D812DFE077E268A410DE4CDEC23FB45EFF7C3C86605BA91BDDE66F82CC1B3D36 Recommended!
Hopefully for the last time as installer software changed.
TC-Archive_tester-0.1.4.0-Setup.rar 0.1.4.0 2017-09-28 819 KB (839.208 bytes) 16C09B2CEF1E62B4B30DB3F205E2003F475DF50E1962EEE785F682F590D4E7D3 No
Link Version Date Size SHA256 hash Requires uninstalling previous version
TC-Archive_tester-0.1.3.2-Setup.rar 0.1.3.2 2017-09-21 815 KB (835.373 bytes) 38E0106AADF15497D55EB6E64D62C333E9B482A2083837DEA5234C460AB82ACC Yes
TC-Archive_tester-0.1.3.1-Setup.rar 0.1.3.1 2017-09-19 820 KB (839.680 bytes) 2068B21BED99B9224D3E4C0A1ECE69CF6D6EEF82F8ED46B489E1642D091CB368 Yes
TC-Archive_tester-0.1.3.0-Setup.rar 0.1.3.0 2017-09-19 815 KB (835.058 bytes) 5647F8D82B59092AE9A1CAF2056A949C1550AEDDC089964CC2716EDA2BFE713A No
TC-Archive_tester-0.1.2.0-Setup.rar 0.1.2.0 2017-09-10 375 KB (384.330 bytes) CB5726D4C46DDA8F888D453BE0CE5E3597821C9FCAC3782F3F1E7DEB2A6BAEDE No

Setup

  1. Download the software package from this page, extract the files and run "setup.exe".
  2. At first run, you have to agree with the licence.
  3. Go to "Settings" tab.
    1. In "Preferences" subtab, the path to the folder where you want logs to be written in "Log folder" field. This is where the result logs will be generated. If it does not exist yet, it will created when the first log file need to be written.
    2. In ".exe files" subtab, fill the paths where "7z.exe" and "flac.exe" can be found. UnRAR.exe should be set already as its license allows its inclusion to this software.
      (You may find 7z.exe through http://www.7-zip.org/ and flac.exe through https://xiph.org/flac/ )

There are some more settings in the "Settings" tab. For now I'll let you explore them.

Usage

For now, you can find some more explainations in the "About" tab.
  • Windows running .NET framework 4.0+
  • 7z.exe (tested with version 16.04 [64] ) to test *.7z, *.bz2 (BZip2), *.gz, *.tar, *.wim, *.xz, *.zip files. Output needs to be in English!
  • flac.exe (tested with version 1.3.2 win32/win64 tested) to test *.7z files. Output needs to be in English!
  • To test *.rar files, UnRAR.exe is provided with the application. Output needs to remain in English!

Version 0.3.0.0 - (Planned, most likely coming before end of summer 2018.)

  • More flexible system to define which tester should be used to test files depending on their type. Which should solve the case of testing multiple volume archives and somewhat solve the case of testing .exe (autoextractible archives) (provided that these .exe file can all be tester by the same tester (unless I implement yet another solution for the .exe files).

Version 0.2.0.0 - (Development started on June 20 2018, coming soon, probably July 2018.)

  • Testing process can now be parallel (multithreaded). Default thread count is based on .NET's "Environment.ProcessorCount" which counts your computer's logical cores. There is a setting to change this manually too.
    Previous version was singlethreaded, relying on the fact the algorithms such as RAR5 are multithreaded. But some, for instance RAR4, are not multithreaded. This should bring a huge performance gain when testing a large number of files encoded with singlethreaded algorithms (or at least when testing algorithms are singlethreaded).
    • "Skip" button is currently inactive when parallel testing, as it is not behaving as expected with multithreading.
  • In the "Inputs: Paths to be tested" field, you may now include/exclude specific files, the same way you would include/exclude a folder.
  • Tests that have been stopped by clicking on "Stop" now count as skipped.
  • Benchmarking: Added "kB/s" indicator
  • There is now a setting to automatically skip some folders by name (separated with "|"). Default value: System Volume Information | $RECYCLE.BIN. (In previous versions, "$RECYCLE.BIN" was hard-coded.)
  • Stability improvement: Closing main form/application now more efficiently closes background threads.
  • UI consistency: "Skip" button is now disabled during the discovery process and only activated during testing process.
  • UI consistency: "Time spent on current file" is unavailable when parallel processing.
  • UI consistency: .exe path text boxes and most setting buttons are now disabled while processing.
  • UI consistency: Progress indicators that can only be evaluated after at least one file has been tested (such as "Est. total time") now read "?" before they can be evaluated, instead of "00:00:00.0000000".
  • Stopping during discovery process will now be much faster (kind of instant).
  • Prevented cases when some file could be tester multiple times or counted multiple times as non testable files (typically if a folder and its parent folder were both set as inputs)
  • At first start, the input text field is now filled with a demo/tutorial, that should be able to scan you all computer if started.
  • UI consistency: "Launch/Testing" button now reads "Scanning" during the scanning processer (instead of "Testing")
  • Possible bug remaining: On my Win7 computer, stopping the testing process ("Stop" button") gets the application stuck with one running parallel test running (according to UI). This did not happen on the Win10 computer where I write the code. Cause unidentified so far.
  • Thinking: Progress indicators work fine is the full process is ran. Using "Skip" or "Stop" buttons result in skipping files, which may make progress indicators interpretation arguable. This doesn't look like a priority but may possibly be re-thought in the future. Probably in a smaller update.

Version 0.1.4.1 - 20180114

  • Fixed a bug which could possibly cause false positives or false negatives in rare cases where specific words appeared in tested file names or paths.
  • Fixed progress bar not being reset when launching tests. (It was re-synching after first file was tested, which made this bug unnoticeable when testing small files.)
  • Fixed a multithreading "bug" which could make some labels not update their value on some systems. However this slightly slows the UI, especially at discovery time, but I think that information accuracy is more important.
  • UI elements such as text fields, checkboxes, radio buttons etc. are now disabled during process.
  • If some tester .exe path is filled in "Configuration" but actual tester .exe file is not found, associated files now go to a new "Tester_not_found" output channel/folder.
  • If some tester .exe path is not filled in "Configuration", files of associated types now go as "Non testable files".
  • Forced 7-Zip output to be UTF-8.
  • Using "Inno Setup" installer instead of "Microsoft Visual Studio 2017 Installer Projects". "Inno Setup" offers more flexibility regarding version numbers, licence agreement, uninstaller, icons, ...
    • In Windows "App & features", version numbers are now shown as x.x.x.x instead of x.x.x.
    • As it is now included in installer, licence agreement as been deactived in the software.
    • "Archive tester" as a new icon. As well as a new uninstaller icon for Start Menu.
    • Hopefully it will not be necessary to uninstall previous versions when installing some new versions in the future.
  • Prevented a rare bug that could happen when using Stop or Skip button, caused by trying to check the state of a process that meanwhile had already finished (only seen in debug mode).
  • Slightly optimised the way the tester .exe files paths are tested if a valid one is not provided already (especially regarding "UnRAR.exe" and secondly it will also look for "WinRAR.exe" standard installation path if "UnRAR.exe" was not found).
    • A temporary feature allows to ignore install version of some type of tester by writing "Do not test" in the associated configuration field, resulting in associated files being "Non testable". (This is temporary because a more generic way of associating files and testers is planned in future versions.)
  • Application setting "Seven7zExePath" renamed to "SevenzExePath". Because that's how I write in the rest of the code.

Version 0.1.4.0 - 2017-09-28

  • Skip button and timeout in "Configuration" > "Settings". Because some large corrupted archives files may take very long, possibly cause a background timeout. Skipped files get removed from progress counts.
  • Ability to exclude paths: Add a "-" (minus) at start of the line to exclude a folder.
  • Progress is now bar based on byte progress instead of file progress.
  • "Settings" tag renamed to "Configuration", new "Settings" tab added under "Configuration" tab.

Version 0.1.3.2 - 2017-09-21

  • Fixed: Bug that could stop file discovery in folder containing non-testable files.

Version 0.1.3.1 - 2017-09-19

  • Fixed a bug at application start due to name change of the licence file.
  • Fixed: Hopefully fixed a possible crash on application close.
  • Fixed: Time spent on current file time label which was not exactly behaving as the other time labels.

Version 0.1.3.0 - 2017-09-19

  • More archive types tested through 7z.exe: *.bz2 (BZip2), *.gz, *.tar, *.wim, *.xz
  • FLAC: *.flac files may now be tested through flac.exe. (You will need to configure flac.exe path in the "Settings" tab.)
  • "Archive tester" is now using UnRAR.exe instead of RAR.exe. UnRAR.exe is included in "Archive tester" package as its license allows it. You may still point to another *.exe file if you wish.
  • In "Settings" tab, you now point at *.exe files instead of a folder + hardcoded *.exe file.
  • Stop button can now stop process while a file is being tested.
  • New "Time spent on current file" timer.
  • Two new log "channels": "Exceptions at discover time" and "Exceptions at testing time". Hence, the exception display form is less often used.
  • Most Exceptions should not stop the process anymore.
  • Application name set to "Archive tester", and "ArchiveTester" as machine friendly name. Exit "TC's Archive Tester" etc.
  • Estimated remaining time label is not shifted to the left anymore.
  • Progress labels now have colors: Green for success, orange for things that may be acceptable but you should give a check, red for failed things.
  • Refactoring for features "extensibility".
  • Application and shortcuts icons should now be displayed correctly.
  • Fixed: UTC checkbox had no effect.
  • License added.
  • ...

Version 0.1.2.0 - 2017-09-10

  • First release on website
  • Fallback log: A secondary window used if, for any reason, log files could not but written (e. g., too long path for Windows)
  • Exception display window: A secondary window that can display possible exceptions/errors
  • ...
  • Process could get broken if .exe files command line outputs change. In that case you would most likely get "Failed" test results for files that are not actually broken. (Which is safer than the other way around.)
  • As 99.99% of my archives are *.rar files, and even my *.flac files are in *.rar archives, RAR is the format I tested the most by simply using it.

As for many things, I see A LOT of ways I can improve this application. Progression will depend on the time I find for it. Obviously, in case it meets success and generates some money through donations, I may give it more attention.

Features and improvements ideas

  • Making treatment by file type more flexible
  • Handling multi-volumes
  • Check if there is a chance that redundant paths get tested multiple times and prevent this to happen
  • What to do with auto-extract *.exe archives?
  • Replacing the licence screen with "Welcome" screen including the licence and some required settings such as .exe file paths
  • Make logs that arenicer to read? HTML? XML/XSLT? JSON/JS?
  • ...

Coding and debugging ideas

  • Use DLLs instead of command line? Do such DLLs able of testing files actually exist? For .NET?
  • For now, it is often necessary to uninstall previous version before installing new version. This is related to version numbers.
  • ...

Support ideas

  • Video tutorial(s)?
  • Adding some screenshots here
  • ...