time33 hash

发表于
本文字数： 3.5k 阅读时长 ≈ 3 分钟

一个好的散列函数通常倾向于“为不相等的对象产生不相等的散列码”。理想情况下，散列函数应该把集合中不相等的实例均匀地分布到所有可能的散列值上。要想完全达到这种理想的情形是非常困难的。幸运的是，相对接近这种理想情形则并不太困难。

由Daniel J. Bernstein教授多年前在comp.lang.c发表的Times 33算法。 它是有史以来发布的最有效的哈希函数之一。

算法介绍
首先，引用一段关于Times 33的介绍：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

DJBX33A (Daniel J. Bernstein, Times 33 with Addition)

 

This is Daniel J. Bernstein's popular `times 33' hash function as
posted by him years ago on comp.lang.c. It basically uses a function
like ``hash(i) = hash(i-1) * 33 + str[i]''. This is one of the best
known hash functions for strings. Because it is both computed very
fast and distributes very well.

 

The magic of number 33, i.e. why it works better than many other
constants, prime or not, has never been adequately explained by
anyone. So I try an explanation: if one experimentally tests all
multipliers between 1 and 256 (as RSE did now) one detects that even
numbers are not useable at all. The remaining 128 odd numbers
(except for the number 1) work more or less all equally well. They
all distribute in an acceptable way and this way fill a hash table
with an average percent of approx. 86%.

 

If one compares the Chi^2 values of the variants, the number 33 not
even has the best value. But the number 33 and a few other equally
good numbers like 17, 31, 63, 127 and 129 have nevertheless a great
advantage to the remaining numbers in the large set of possible
multipliers: their multiply operation can be replaced by a faster
operation based on just one shift plus either a single addition
or subtraction operation. And because a hash function has to both
distribute good and has to be very fast to compute, those few
numbers should be preferred and seems to be the reason why Daniel J.
Bernstein also preferred it.

-- Ralf S. Engelschall rse@engelschall.com

理解：

Times 33是Daniel J. Bernstein多年前在comp.lang.c上发表的哈希算法，这个算法已被广泛应用，是目前最好的字符串哈希算法之一。因为它不仅计算速度很快，而且分布比较均匀。

核心逻辑是这段代码：

1
hash(i) = hash(i-1) * 33 + str[i]

这个神奇的数字33，为什么用来计算哈希的效果会比其他许多常数（无论是否为质数）更有效，并没有人给过足够充分的解释。因此，Ralf S. Engelschall尝试通过自己的方法解释其原因。通过对1到256中的每个数字进行测试，发现偶数的哈希效果非常差，根据用不了。而剩下的128个奇数，除了1之外，效果都差不多。这些奇数在分布上都表现不错，对哈希表的填充覆盖大概在86%。

从哈希效果来看（Chi^2应该是指卡方分布），虽然33并不一定是最好的数值。但17、31、33、63、127和129等相对其他的奇数的一个很明显的优势是，由于这些奇数与16、32、64、128只相差1，可以通过移位（如1 << 4 = 16）和加减1来代替乘法，速度更快。

算法实现
DJB Hash Function
An algorithm produced by Professor Daniel J. Bernstein and shown first to the world on the usenet newsgroup comp.lang.c. It is one of the most efficient hash functions ever published.

1
2
3
4
5
6
7
8
9
10
11
12
unsigned int DJBHash(const char* str, unsigned int length)
{
   unsigned int hash = 5381;
   unsigned int i    = 0;

   for (i = 0; i < length; ++str, ++i)
   {
      hash = ((hash << 5) + hash) + (*str);
   }

   return hash;
}

http://www.partow.net/programming/hashfunctions/

djb2
this algorithm (k=33) was first reported by dan bernstein many years ago in comp.lang.c. another version of this algorithm (now favored by bernstein) uses xor: hash(i) = hash(i - 1) * 33 ^ str[i]; the magic of number 33 (why it works better than many other constants, prime or not) has never been adequately explained.

1
2
3
4
5
6
7
8
9
10
11
unsigned long
hash(unsigned char *str)
{
    unsigned long hash = 5381;
    int c;

    while (c = *str++)
        hash = ((hash << 5) + hash) + c; /* hash * 33 + c */

    return hash;
}

http://www.cse.yorku.ca/~oz/hash.html

参考
http://www.partow.net/programming/hashfunctions/

http://www.cse.yorku.ca/~oz/hash.html

https://en.wikipedia.org/wiki/Universal_hashing

《Effective Java中文版本》第2版

转载来源：http://zhanjia.iteye.com/blog/2426782