Integrity

SHA-1

SHA-1 (“Secure Hash Algorithm #1″, also “SHA1″) is the second most common data integrity check standard (a.k.a. “hash”) used throughout the world today.  SHA-1 codes are 160-bit numbers and are usually represented in hexadecimal format (e.g., “de9f2c7f d25e1b3a fad3e85a 0bd17d9b 100db4b3″).

SHA-1 is the least secure hash algorithm NIST currently supports in its FIPS validated cryptography implementations.   However, SHA-1 is faster than SHA-2 (its successor) and is commonly available in file transfer technology today (e.g., FTP’s unofficial “XSHA1″ command).

BEST PRACTICE: Modern file transfer deployments should use FIPS validated SHA-1 or SHA-2 implementations for integrity checks.  Some SHA-1 usage is already prohibited in high security environments.  If you are working with a U.S. Federal Government application, read NIST’s “Transitioning of Cryptographic Algorithms and Key Sizes” to see if your SHA-1 implementation is still allowed.

SHA-2

SHA-2 (“Secure Hash Algorithm #2″) is the most secure hash algorithm NIST currently supports in its FIPS validated cryptography implementations.  SHA-2 is really a collection of four hashes (SHA-224, SHA-256, SHA-384 and SHA-512), all of which are stronger than SHA-1.

Complete SHA-2 implementations in file transfer are still uncommon but becoming more common as time passes.

A “SHA-3” contest to find a future replacement for SHA-2 is currently underway and expected to conclude in 2012.

BEST PRACTICE: Modern file transfer deployments should use FIPS validated SHA-1 or SHA-2 implementations for integrity checks.

SHA-3

SHA-3 refers to the new hash algorithm NIST will choose to someday replace SHA-2.   A contest to select the new hash is scheduled to conclude in 2012.

SHA-384

SHA-384 is the 384-bit component of the “SHA-2” data integrity check standard (a.k.a. “hash”).  It is not a unique hash algorithm within the SHA-2 standard but is instead a truncated version of SHA-512.

See “SHA-2” for more information.

SHA-224

SHA-224 is the 224-bit component of the “SHA-2” data integrity check standard (a.k.a. “hash”).  It is not a unique hash algorithm within the SHA-2 standard but is instead a truncated version of SHA-256.

See “SHA-2” for more information.

SHA-512

SHA-512 is the 512-bit component of the “SHA-2” data integrity check standard (a.k.a. “hash”).  Like SHA-256, it is one of two unique algorithms that make up a SHA-2 hash, but SHA-512 is optimized for 64-bit calculations rather than 32-bit calculations.

See “SHA-2” for more information.

SHA-256

SHA-256 is the 256-bit component of the “SHA-2” data integrity check standard (a.k.a. “hash”).  Like SHA-512, it is one of two unique algorithms that make up a SHA-2 hash, but SHA-256 is optimized for 32-bit calculations rather than 64-bit calculations.

See “SHA-2” for more information.

MD4

MD4 (“Message Digest [algorithm] #4″) is best known as the data integrity check standard (a.k.a. “hash”) that inspired modern hashes such as MD5, SHA-1 and SHA-2.  MD4 codes are 128-bit numbers and are usually represented in hexadecimal format (e.g., “9508bd6aab48eedec9845415bedfd3ce”).

Use of MD4 in modern file transfer applications is quite rare, but MD4 can be found in rsync applications.  A variant of MD4 is also used to tag files in eDonkey/eMule P2P applications.

Although MD4 is considered a “cryptographic quality” integrity check (as specified in RFC 1320), it is not considered a secure hash today because it is possible for an attacker to create bad data that bears the same MD4 code as a set of good data.  For this reason, NIST does not allow the use of MD4 in key U.S. Federal Government applications.

BEST PRACTICE: Modern file transfer deployments should use FIPS validated SHA-1 or SHA-2 implementations for integrity checks instead of MD4.

MD5

MD5 (“Message Digest [algorithm] #5″) is the most common data integrity check standard (a.k.a. “hash”) used throughout the world today.  MD5 codes are 128-bit numbers and are usually represented in hexadecimal format (e.g., “9508bd6aab48eedec9845415bedfd3ce”).

MD5 was created in 1991 as a replacement for MD4 and its popularity exploded at the same time use of the Internet did as well.  MD5′s use carried over into file transfer software and its use remains common today (e.g., FTP’s unofficial “XMD5″ command).

Although MD5 is considered a “cryptographic quality” integrity check (as specified in RFC 1321), it is not considered a secure hash today because it is possible for an attacker to create bad data that bears the same MD5 code as a set of good data.  For this reason, NIST has now banned the use of MD5 in key U.S. Federal Government applications.

BEST PRACTICE: Modern file transfer deployments should use FIPS validated SHA-1 or SHA-2 implementations for integrity checks instead of MD5.  However, FTP software that supports the XMD5 command can be useful to provide backwards compatibility during migration to stronger hashes.

CRC

CRC (“cyclic redundancy check”) is an early data integrity check standard (a.k.a. “hash”).  Most CRC codes are 32-bit numbers and are usually represented in hexadecimal format (e.g., “567890AB”).

CRC was commonly used with modem-based data transfer systems because it was cheap to calculate and fast on early computers.   Its use carried over into FTP software and some vendors still support CRC in their applications today (e.g., FTP’s unofficial “XCRC” command).

However, CRC is not considered a “cryptographic quality” integrity check because it is trivial for an attacker to create bad data that bears the same CRC code as a set of good data.

BEST PRACTICE: Modern file transfer deployments should use FIPS validated SHA-1 or SHA-2 implementations for integrity checks instead of CRC.  However, FTP software that supports the XCRC command can be used to supplement stronger integrity checks, particularly over unreliable connections.  (i.e., If your application calculates a bad CRC code for a particular transfer, you can avoid the effort of calculating a more expensive SHA-1 or SHA-2 hash.)

Event Log Analyzer by SolarWinds