
Click here for full text:
Relations Between Random Coding Exponents and the Statistical Physics of Random Codes
Merhav, Neri
HPL2007149
Keyword(s): random coding; free energy; partition function; phase transitions; error exponents
Abstract: The partition function pertaining to finitetemperature decoding of a (typical) randomly chosen code is known to have three types of behavior, corresponding to three phases in the plane of rate vs. temperature: the ferromagnetic phase, corresponding to correct decoding, the paramagnetic phase, of complete disorder, which is dominated by exponentially many incorrect codewords, and the glassy phase (or the condensed phase), where the system is frozen at minimum energy and dominated by subexponentially many incorrect codewords. We show that the statistical physics associated with the two latter phases are intimately related to random coding exponents. In particular, the exponent associated with the probability of correct decoding at rates above capacity is directly related to the free energy in the glassy phase, and the exponent associated with probability of error (the error exponent) at rates below capacity, is strongly related to the free energy in the paramagnetic phase. In fact, we derive alternative expressions of these exponents in terms of the corresponding free energies, and make an attempt to obtain some insights from these expressions. Finally, as a side result, we also compare the phase diagram associated with a simple finitetemperature universal decoder, for discrete memoryless channels, to that of the finitetemperature decoder that is aware of the channel statistics.
26 Pages
Back to Index
