Lecture 24: Linear Gaussian Channels

• Download English-US transcript (PDF)

PROFESSOR: Today we finish up chapter 13. I think we are finished, but I'll take questions. We get into chapter 14, which starts to get into coding for bandwidth-limited channels. Here we're coding directly in terms of Euclidean space rather than in terms of Hamming space. And there's this wonderful quote from Neil Sloane. "Euclidean space coding is to Hamming space coding as classical music is to rock'n'roll." Meaning that, in the Euclidean space, things are continuous, whereas we were discrete back in Hamming space.

Nonetheless, there are strong connections between the two, which in two lectures I'll basically only have a chance to hint at. You were supposed to hand in problem set nine last Friday, or today if you haven't already. Today, we'll hand out the solutions for nine, and then chapter 14 and its problems, which are problem set 10. These are not due. As always, I recommend you do them. The solutions will be handed out Wednesday.

However, on the exam, the exam will not cover chapter 14. So everything we do this week is gravy. The exam is a week from tomorrow in the morning. Ashish will administer it because I won't be here. For this exam, it's closed book, except you're allowed to make five pages of notes, as you did three pages on the midterm. And I do recommend you bring a calculator, because for one of the problems the calculator will be helpful. And I was told that erasable memory should be cleared on the calculator, so point of honor if you do that.

Any questions about the exam or chapter 13? Yes?

AUDIENCE: Are you including the midterm [UNINTELLIGIBLE]?

PROFESSOR: Yeah, oh, everything in the course up to chapter 13. The rule is everything that we covered in class. If we didn't cover it in class, then you're not responsible for it. Other questions? Any last questions on chapter 13? This is kind of the finale for binary codes. We found that, at least on the binary erasure channel, we had techniques that would allow us to design LDPC codes that would get us as close to capacity as we liked, or at least I made that assertion, directed you to where you could find that done in the literature.

And I further asserted that, at least for binary symmetric input channels, like the binary additive white Gaussian noise channels, you could do much the same thing using techniques like density evolution or EXIT charts. Again, the issue is to design code, such as this iterative decoding, after many, many, many iterations, will, with probability one, converge to the correct solution (or with probably one minus epsilon).

So that's the great conclusion to the story for binary codes, channels in which we want to have binary inputs. And way back in the beginning, we said that that was perfectly sufficient, really, for the general additive white Gaussian channel. When we were talking about binary code rates less than 1/2, or nominal spectral efficiencies less than one bit per second per Hertz, which are equivalent statements.

So what happens if we want to go over channels where we want to send at higher spectral efficiencies? We want to send four bits per second per Hertz, as we certainly do over, say, telephone line channels, which are 3,000, or 4,000-Hertz channels, over which we want to send tens of thousands of bits per second. Or many radio channels will support high numbers of bits per second per Hertz, and so forth. Well, obviously, you can't do that with binary codes.

So, today and Wednesday in this chapter, we very quickly, of course, go through the series of techniques that have been developed to go over bandwidth-limited channels such as the bandwidth-limited additive white Gaussian noise channel. Clearly, you're going to need a non-binary constellation, and somehow code on that constellation.

But what you'll see is that both the techniques and much of the development are very reminiscent of the binary case. Historically, the binary case was always done first, and then the generalizations to non-binary followed. And so we'll see -- you can start off with very simple, hand-designed constellations, as we were doing back in the early chapters, four and five. We played around with little m equals eight constellations and tried to optimize them from the point of view of, say, minimizing power for a fixed minimum distance between points in a certain number of dimensions in Euclidean space.

Now, we want to go up to considerably longer constellations. So that was kind of at the level of Hamming codes and single parity-check codes and very elementary binary coding techniques like that. So we want more complicated codes, let's say moderate complexity codes. We will find things that are analogous to block codes, linear block codes, namely lattices. These are structures in Euclidean space that have the group property, as linear block codes do. And then we'll find trellis codes, which are the Euclidean space analog of convolutional codes.

And basically, our goal in the next two lectures is to get to the level of a union-bound estimate analysis of the performance of these kinds of coding techniques, which, like their binary analogs, sort of sit in a moderate complexity, pretty good performance, get up to 6 dB coding gain, can't get to capacity. So that's as far as we'll be able to go in chapter 14.

In the last page or two of chapter 14, I briefly mention higher performance techniques. And it's generally accepted that you can get as close to capacity on these bandwidth-limited channels, additive white Gaussian noise channels as you can on the binary channels, in some cases just by adapting binary techniques in a kind of force-fit way, and others that are somewhat more principled, I might say.

But in any case, by adapting the binary techniques, you can get turbo codes, low-density parity-check codes, capacity approaching codes of various types for the bandwidth-limited channel. But I just hint how that can be done in one page, and I can't include it in the course. So that's where we're going.

So basically, we're trying to design a constellation set of signal points. Constellation c or signal set or alphabet, script c, if you like. And we're doing the kind of things we were doing back in chapter four and five. What are some of the things that the parameters we have-- we're going to do this in n-dimensional space. We're going to be concerned with the size of the constellation, the log of the size, the number of log 2 of the size, the number of bits we can send.

We call this m at one point. That will determine the number of bits we can send per n dimensions, or we can normalize it per two dimensions. Typically, I said, in bandwidth-limited regime, we'll normalize everything per two dimensions. We're going to have an average power of the constellation, which I'm just going to write like that. We're going to have a minimum squared distance between points in the constellation, which I'm going to write like that.

And the idea is to optimize the trade-off between this and this. Clearly, if I take a given constellation and I scale it up, I'm going to get better minimum squared distance without the cost of increasing power. So I either fix the minimum squared distance between points and try to minimize the power, the average energy over the centroids of a bunch of spheres, or vice versa.

So when we do this, this is kind of the game, we get into a classical problem called sphere-packing. If we say, hold the minimum squared distance constant, that means we want to have a certain minimum distance. Here I'll make it d_min over 2. That means that there's going to be a sphere of radius d_min over 2 around each point in this space, which must not be violated by the sphere of any other point. So basically, we're trying to pack spheres as closely together as we can. And we're maybe given the dimension. Here I have two dimensions to play with. We may be given the number, m.

In general now, we're going to be thinking of m becoming very large. We want to go to arbitrarily large number of bits per second per Hertz. So think of schemes that will continue to work as we add more and more points. And if I do that in two dimensions, we get penny-packing. Take a bunch of seven pennies out of your pocket, and try to squeeze them together as tightly as you can. That's effectively what we're trying to do. They maintain their fixed radius, and you get something that quickly starts to look like what this is called, a hexagonal sphere-packing, because the centers line up on a hexagonal grid.

And in fact, I forget exactly what we did. We did some of these back in the early chapters. And let me draw a 16-point constellation, which I remember we did draw. Here's a constellation consisting of 16 pennies. And subject to my drawing ability, it's packed as closely as possible for a fixed minimum distance between the points.

And as far as anyone knows, this constellation, consisting a set of points on a hexagonal grid, is the most dense packing of 16 points in two dimensions, as measured by the average power of the centers. And it's not very symmetrical, but you can see it's based on a symmetrical packing. This hexagonal, this is basically a subset of 16 points from the hexagonal lattice. If we extend these points in all directions-- again, I haven't done it very well-- but the hexagonal lattice is simply the underlying lattice here.

Let me just draw a set of points that look like this. I'm going to fail again. Sorry, I'm better at rock'n'roll than I am at classical music. Anyway, this is the lattice, and it's called hexagonal because, if we draw the decision regions for this lattice, if we consider this as a set of points in a constellation and draw the decision regions, it looks like that, this little hexagon, defined by the six nearest neighbors. They define six sides of a hexagon.

And so each lattice point has a little region of space that belongs to it, you can think of as its decision region. Now, what makes this a lattice? What's the definition of a lattice? We'll say an n-dimensional lattice. Very simple and elegant definition for a lattice. It's a discrete subset. That means a set of discrete points of Rn, sitting in Euclidean n-space, that has the group property. And you now all remember what that means. It means the group property, under ordinary addition of vectors in Euclidean n-space, addition or subtraction, of course.

What are some of the implications of having a group property? That means that the sum of any two vectors is another vector in the lattice. Does the lattice have to include the 0, the origin point? Anybody? Oh, good. I'm going to ask this on the final. Come on. Does it or doesn't it?

AUDIENCE: It does.

PROFESSOR: It does. Why?

AUDIENCE: [INAUDIBLE].

PROFESSOR: OK, that's one good answer. It has to be closed under addition and subtraction. Subtraction of the point from itself is going to give you 0. A more mathematically trained person would probably, say, well, it obviously has to have an identity. Every group has an identity element. And that means, if lambda is in the lattice, does minus lambda have to be in the lattice? Yeah, every element has to have an additive inverse, and so forth.

Well, so if I have a lattice like this, that means that one of these points has to be the origin, say this one. Now, of course, you can have a translate of a lattice, and geometrically it has all the same properties as the lattice itself, at any fixed t to all the lattice points. In other words, make the origin somewhere else. But that's not itself a group. That's the co-set of a group. It would translate as a co-set operation.

So lambda must be a group, and lambda plus t, any lambda translate is a co-set of lambda. So most of the things you say about lambda are also true for lambda plus t, but in order to fix a group, we need the origin to be part of the points set. OK, if it's a group, what does that mean? That means, if we take any particular point, say this one, regard it as a vector, then call this lambda_1 -- or let me it g1, more suggestively. Does g1 plus g1 have to be in the group? Yes.

So that forces this point to be in there, 2g1, 3g1, so forth. All of these have to be in there, as well as minus g1, minus 2g1, and so forth. So basically, once we found a generator, all integers times that generator have to be in the lattice. So that means there's a cross-section along the axis defined by g1, such that these points are simply the integers, scaled by whatever the norm of g1 is.

And of course, that holds for any of these neighbors here. You want to look for nearest neighbors of 0's. Take g2 and then, of course, all of its integer multiples have to be in the lattice. So we've got a set here that, in some sense, spans n-space. We have now two vectors, g1, g2, which are not linearly dependent. They're sitting in 2-space. So if we take all real linear combinations of these two vectors, we're going to get all of 2-space.

If we take all integer linear combinations of these two vectors, they all must be in the lattice, right? By the group property? We've already proved that all the integer multiples of either of them is in the lattice, so any thing plus or minus those two. So certainly, the lattice includes the set of all integer multiples of a_i times gi, where i equals 1 to 2, where a1, a2 is in the two-dimensional integer, lattice Z-squared.

Could there be any other points? There actually could, if you made a mistake and picked your initial vector to be 2g1, because then you wouldn't have taken account of g1. So you've got to take your two initial vectors to be two of the minimum norm points in the lattice. But if you do that, then you can easily prove that this is all of the lattice points.

So in fact, a two-dimensional lattice is equal to a set of all integer linear combinations of two generators, which we can write as a little two-by-two generator matrix, G. Or very briefly, we could write lambda as G times Z-squared, and we can view it as just some linear transformation of Z-squared.

So in this hexagonal lattice, we can view it as starting out with a square grid. Of course, Z-squared itself is a lattice. Z-squared is a group. This is just the square lattice in two dimensions. We have Z_n as a lattice in n dimensions, in general. So I take Z-squared, Z_n, more generally, and I distort it. I just run it through some g, which is a linear transformation, and slants some of the generator points.

And that's a general way of getting a lattice. That's not the way we usually visualize the hexagonal lattice. We usually visualize it in such a way as to recognize that it has six-fold symmetry. But that's certainly a way to get it. And this has one interesting consequence. One of the measures we want for a lattice is the volume per lattice point. In here, the volume, you can divide this up into volumes that are all of equal size, and we call that the volume of A2. This lattice -- hexagonal lattice, is called A2, and it's the volume of that hexagon.

1 over the volume of that hexagon is the density of points in 2-space, if you like. So density is 1/volume. I mean, think of every point as having a unique volume, and if we basically consider the cells around each lattice point, they fill up 2-space, or the perfect tessellation tiling of 2-space. These hexagons will.

One easy consequence of the group property is that all the decision regions have to be congruent, in fact, just translates of one another by lattice points. The decision region around this point is the same as the decision region about 0, translated by a lattice point, namely the center of that decision region. I won't prove that but it's true and easy to prove.

So that's one way of finding what's the volume. The volume will mean the volume of 2-space per lattice point. But what's another way to do that? Another way is just to take this little parallelotope, whose volume is easier to calculate. And we can see that we can equally well tile 2-space with these little parallelograms, just anchor -- in this case, this is the one that's anchored in the lower left corner by the origin.

Then we take another one that's anchored by this point. We take another one that's anchored by this point, another one that's anchored by this point. And it's clear that's another tiling of 2-space by regions, one for each lattice point that completely fills 2-space with overlaps only on the boundary, which don't count.

So the volume of A2 is also the volume of that, what's called a fundamental parallelotope. And what's the volume of that fundamental parallelotope? Well, one way of calculating it is to take the volume of Z-squared. We can view this as the distortion of Z-squared, which looks like this. So if this is Z-squared, what's the fundamental volume of Z-squared? The volume of Z-squared per lattice point. This is (0,1,-1,1, so forth,1,1). What's the volume of Z-squared per--

AUDIENCE: 1

PROFESSOR: --1. Clearly. And we have this decision region. This is little square side 1, or this parallelotope is also little square side 1. It's volume is 1. General geometric principle, if we take this and distort it by G, then what's the volume of, say, this parallelotope going to be? This parallelotope goes by G into this one, and its volume is the Jacobian, which is the determinant of G.

So the volume of a lattice is the determinant of any generator matrix for that lattice. Just a nice little side fact, but it's an example of the kind of things you get in Euclidean space that you don't get in Hamming space. Of course, there are various generator matrices that you could use to generate this lattice. But whichever one you choose, this is going to be true.

So this is an invariant among all the generating matrices. So that's a lattice. I haven't mentioned what's the most important thing for us about a lattice. A lattice is a group. What's the fundamental property of a group, which I guess was embodied in the alternate set of axioms for the group? If I take the group plus any element of the group, what do I get?

AUDIENCE: [INAUDIBLE].

PROFESSOR: The group again. Let's interpret that geometrically. One of the nice things about this subject is you're constantly going back and forth between algebra and geometry. Geometrically, what does that mean? That means, if we take this lattice, say the origin is here, and we shift it, say, so that the origin is up here, then nothing changes. Everything is invariant to this shift. We can do this for any lattice point.

That shows you that all the decision regions have to be congruent. It shows you that the number of nearest neighbors, the minimum distance from any point to its nearest neighbors is always the same for any lattice point. Basically -- it's a lattice is geometrically uniform. That means you can stand on any of the points and the world around you looks the same, as if you stood on any of the other points.

Think of these as stars in the universe, and somebody blindfolds you and drops you on one of these points. Can you tell which one of those points you've been dropped on if nobody's written anything on these points? No, you can't tell. The world looks the same from whichever one you drop on. So that means the minimum square distance from any point is the same as every other.

So the number of nearest neighbors at minimum square distance is the same. It means the total distance profile, starting from any point, is the same. It means if we were to take one of these points and send it over a Gaussian channel from a very large set, and we take one of the points from the interior and send it over an additive white Gaussian noise channel, the probability of error is going to be the same, regardless of which point we send it from, because the probability of error is the probability of falling outside your decision region. And since the decision region always has the same shape, it's invariant to which point you send.

So this geometrical uniformity property, which that's the general name for it and it has all these specific consequences that I just listed, is the principal geometrical property of a lattice. This just follows from its group property. You don't have to be a lattice to have this property. For instance, the translate of a lattice is not a group, technically, but it obviously has the same geometrical uniformity property. And there are more elaborate examples of things that look less like lattices that are nonetheless geometrically uniform. On the other hand, lattices definitely are geometrically uniform.

So this seems to be a good thing to do, because what's our objective? We're trying to have d_min-squared be the same for every point in our constellation. We want to tack these pennies as close together as we can. That means we want to hold d_min-squared constant for every point that we're going to use. And so lattices have that property. So that seems like a good way to do sphere-packing.

And in fact, sphere-packing, in n-dimensions, is a very old mathematical subject. How many spheres can you pack around a three sphere, is it 12 or 13? That was debated in the 18th and 19th centuries, and I think it's only finally just been resolved a few years ago. This turns out to be not an easy problem. The densest sphere-packing in four dimensions was found in the mid-19th century. In eight dimensions, I think around the turn of the 20th century. In 16 dimensions, it's the Barnes-Wall lattice, which we'll see a little later, which was found in '54, about the same time as Reed-Muller codes, or maybe it was '59, maybe a little later than Reed-Muller codes.

The Leech lattice was certainly found a little earlier than that, but around the same time as the Golay code, to which it's a very close cousin. And so forth. So you might think that these questions had all been settled, but in fact they're still open for most specific dimensions, n. Conway and Sloane have the authoritative book on this, which basically lists what we know about the densest sphere-packings in all dimensions.

So here's our idea. Our idea is we're going to go consult the mathematicians and find out what the densest sphere-packings that they found are in various dimensions. In two dimensions, it's the hexagonal lattice. In one dimension, it's the integer, and basically the integers are the only packing with regularity properties in one dimension. And two dimensions, you could consider Z-squared, or you can consider A2. These are the two obvious candidates, the square and the hexagonal packings.

In three dimensions, you get face-centered cubic packing. And what's the BCC? The base-centered cubic packing, I guess.

AUDIENCE: Body-centered.

PROFESSOR: Say again.

AUDIENCE: Body center.

PROFESSOR: Body-centered, thank you. I'm sure you're right. So anyway, there's some things that are known about this from classical mathematics and current mathematics. So our idea is we're going to try to find the densest possible lattice in n dimensions. But for digital communications, for coding, we only want a finite number of points from that constellation. So what we're going to do is we're going to create our constellation based on a lattice and on a region of n-space. And the idea is simple. We're just going to intersect the lattice. Or maybe let me allow a translate of a lattice with the region.

And make this a finite, compact region. And this will give us a certain finite subset of the points in the lattice. Now, when I draw a constellation like this one, of a size 16, that's what I've done. I've basically taken A2. Or it's actually a translate of A2. I think the origin is in here somewhere. And I draw a sphere around the origin. And I take the 16 points that happen to fall within that sphere. And I call that my constellation.

So that's the idea in general. Think of the region, r, as a cookie cutter. We're going to take the cookie cutter and chop out a finite number of points from this nice, regular, infinite grid. And that will give us a constellation. So especially for large constellations, we'll be able to make some approximations that will allow us to analyze that kind of constellation very easily.

And actually, the best constellations that we know are of this type. We've taken this baseline, m-PAM. Well, what's m-PAM? We take the lattice plus t, let's say to be Z plus 1/2. So that means we don't take the origin, but we take the points 1/2 minus 1/2, 3/2. They're equally spaced points, but we've shifted them so as to be symmetric about the origin. 5/2 minus minus, so forth. So this is Z plus 1/2 is our lattice. And then if we want 4-PAM, we take a cookie cutter consisting of this region.

So this is the region going from minus 2 to 2. It's not unreasonable to expect that, if we take a region of length 4, we're going to get 4 points, if, again, it's symmetric around the corner, because each of these points takes up about one unit of space over here. So if we made it from minus 4 to plus 4, we'd have a region of length 8, and we'd get 8 points. And so forth. So that's how we do m-PAM, and then scale it if we want to. But this is the basic idea.

Similarly, for m by m QAM, like everything else, it's just doing the same thing independently in two dimensions. Here we take our lattice to be Z plus 1/2 squared, which is just the offset Z-squared, offset so as to be four-way, to be roughly symmetrical to four-way rotations, 90-degree rotations.

So here is Z-squared plus (1/2, 1/2). And then, again, here to get 16-QAM, for instance, we might take the region to be what? (-2,2)-squared which, again, has area 16. You might think of this as just taking the little square around each of these. Each of these squares has area 1. So when we take the union of all those squares, we get the region.

That's another way of characterizing this region. It's just the union of all the decision regions of the points that we want. If you had started from the points, you could certainly invent a region that is the union of those decision regions. It would have an area equal to the number of points times the volume of 2-space per each point.

That's the kind of approximation we're going to use. The bounding region is clearly going to have something like the number of points in the constellation, times the fundamental volume as its volume. So this is our idea. This is what we're going to call a lattice constellation. And we're going to investigate the properties of lattice constellations, not the most general thing we could think of but good enough.

Seem to have the right sort of property. And we're going to take it at least as far as the union-bound estimate, so we can get a good estimate of probability of error if we take one of these constellations, like m-by-m QAM, and use it to transmit over a Gaussian noise channel. And furthermore, for large constellations, meaning as we get well up into the bandwidth-limited regions of large nominal spectral densities, we're going to be able to separate the analysis into lattice properties and region properties. And we'll basically be able to just add these up to get the overall analysis, by making some judicious approximations that become exact in the large constellation limit.

So let's now restrict ourselves to lattices, or translates of lattices, and start to analyze them. What are their key parameters for our application, which is digital communications? Well, one is the minimum squared distance between lattice points. We've seen that doesn't depend on the particular point. So this is the minimum squared distance from any lattice point. The number of nearest neighbors. The volume per lattice point. And is there anything else? It seems to me there's a fourth one. No. I guess that's all we need to know.

So I want to combine these into a key figure of merit for coding. Again, we did this back in chapter four for a complete constellation. We found the figure of merit by holding the average power fixed for a fixed number of points, m, on a fixed dimension, n, and maximizing the minimum square distance, or, conversely, holding the minimum squared distance fixed and minimizing the average power. So we get some figure of merit that was kind of normalized, and we could just optimize that one thing.

The key figure of merit was called by 19th-century mathematician, Hermite's parameter, but we are going to call it the nominal coding gain because that's what it's going to turn out to be. And it's just defined as follows. The nominal coding gain of a lattice is just this minimum squared distance, so it goes up as the minimum squared distance. But then we want to normalize this. And the obvious thing to normalize it by, we want something that's going to scale as a two-dimensional quantity. If the lattice was scaled by alpha, then this would go up as alpha squared. So we want something that'll scale in the same way.

And what we choose to normalize it by is the volume of lambda per two dimensions, if you like. So you can see immediately, this is invariant to scaling, that the nominal coding gain of lambda is the same as the nominal coding gain of alpha lambda. Is alpha lambda a lattice, by the way? If lambda's a group, and we have any scale factor alpha greater than 0, then alpha times lambda is also a group. Everything's just multiplied by alpha. So alpha lambda is a lattice.

And this is actually one of the homework problems. If you scale everything by alpha, d_min-squared goes up by alpha squared. The volume goes up by alpha to the n, for an n-dimensional lattice. But when we take 2 over n, we're going to get back to alpha squared. So alpha c of lambda is alpha c of alpha lambda. So this is what you want. You want a kind of scale-free version of a lattice, because, obviously, when we use this, we're going to feel free to amplify it or compress it any way we want, so we do with QAM.

All right, so this is normalized in that way. We prove in the homework it's also invariant to other things. If you took, say, an orthogonal transformation of this lattice, a rotation, reflection, multiply it by a matrix that doesn't change the scale, think of it geometrically as an orthogonal transformation, is that going to be a lattice? It is going to be a lattice. Everything's just going to have some h out here, times G times Z-squared, and that's still of the form of a lattice. Or call it U for orthogonal, unitary.

So we multiply it by some orthogonal matrix, U. It's still a lattice. If we look in here, orthogonal transformations don't change squared distances. We're not going to change this -- it doesn't matter. Actually, I might. No. It's a rigid rotation, I'm sorry. Everything here stays the same. Hexagonal lattice still is hexagonal lattice, because it's a rigid rotation or reflection geometrically. And the volume clearly stays the same, so none of these things changes. And orthogonal transformation isn't going to change this quantity either.

So we can rotate this or do whatever. And then, finally, there's an even more interesting one, which is, if we take the lattice, lambda to the m, this is simply the Cartesian product. It's the set of all lambda_1, lambda_2, lambda_m, such that each of these lambda_k is in Lambda. So it's the set of all n-tuples of elements of Lambda. This clearly lives in dimension mn, so it's a much higher-dimensional lattice. The simplest case is Z to the m - it lives in m dimensions, whereas Z is down in one dimension.

So it lives in dimension mn. It's a way of constructing high-dimensional lattices from low-dimensional lattices. It's something that, in communications, we consider to be almost a non-event. Sending a sequence of elements of lambda is the same as sending a sequence of elements of lambda_m. In fact, if we just send a sequence of things from here, and a sequence of things from lambda, there's no difference from a receiver's point of view. So we regard these two lattices as equivalent, from a communications point of view.

Well, what happens to d_min-squared? d_min-squared doesn't change, because we can always-- d_min-squared is, by the way, the weight of the smallest-- should have said that. You can always do things relative to 0. d_min-squared, therefore, is the squared norm or lowest squared energy of any non-zero lattice point, same as we did with Hamming codes, just by the group property. K_min is the number of such lowest Euclidean weight points.

OK, here, what's the lowest weight point? It would look something like this, where this is one of the lowest weight points in lambda. So d_min-squared doesn't change. This changes, but we're not considering that. The volume, what is the volume? It turns out that it's not hard to show that V(lambda to the m) is V(lambda) to the m. And therefore, you can show that this is figure of merit doesn't change, even if you take Cartesian products.

So for instance, I chose that the nominal coding gain of to the m is equal to the nominal coding gain of Z. Which is what, by the way? I should have done this as the first example. What's the minimum squared distance of Z? 1. What's the volume of Z? 1. So this is 1, or 0 dB. We're going to measure nominal coding gain in dB, which, well, we're getting us into communications.

What's the nominal coding gain of the hexagonal lattice? By asking this question, I'm basically asking, is the hexagonal lattice denser than Z-squared in two dimensions than the square lattice in two dimensions? And if so, quantitatively, how much denser is it?

AUDIENCE: [INAUDIBLE].

PROFESSOR: OK, well, there's a root 3 in it, but that isn't exactly the answer, I don't think. There are various ways that you can do it, but one of the ways is to take, as a generator matrix, (1, 0; 1/2, root 3 over 2). You like that as a generator matrix for the-- That's taking this and this as the two generators, and recognizing this as 1/2, and this as root 3 over 2. That makes the length of this equal to 1/4 plus 3/4, equals 1 square root, and it's 1.

So these are my two generators of length 1 in a hexagonal lattice of minimum squared distance 1. So based on that, I can say that the volume of A2 is what?

AUDIENCE: [INAUDIBLE].

PROFESSOR: Root 3 over 2. Not hard to take the determinant of that, right? And what's the minimum squared distance in this scaling? What's the minimum non-zero weight -- squared norm, it's 1. So this is equal to 1 over root 3 over 2, which is equal to 2 over root 3. Which is equal to what? This is 3 dB. This is 4.8 dB, roughly. Square root is 2.4 dB. So this is about 0.6 dB.

That's where it pays to be facile with dB's. This area, by the way, if you tried to compute the area of this little hexagon here, what would it be? If the area of this parallelotope is the square root of 3/2, what's the area of this hexagon? The same, right? They're both space filling when centered on lattice points. So they have to have the same area. So that's an easy way of computing the area of a hexagon, or of this particular hexagon anyway.

But you would get the same result in either way. So comparing this with Z-squared, I now have a quantitative measure of how much denser A2 is than Z-squared. Put in one way, if I fix the minimum squared distance of each of them to be 1, if I'm doing penny-packing and I say I want pennies of a certain radius to be packed into two-dimensional space, then this is 1 over the volume, which is basically the density. The density of A2 is 2 over root 3, times [UNINTELLIGIBLE] as the density of points in z squared. I get more points per unit area with the same distance between them using A2 than I can with Z-squared.

So that's the significance of this parameter. So it's an obvious thing for Hermite to have come up with. It's going to turn out to be equally fundamental for us. So that may be all we want to do on lattices for the time being. Yeah, seems to be.

So we're talking about lattice constellations. The other thing we have to develop is the properties of regions. So let me talk about regions, R -- I'm thinking of a compact region in an n-dimensional space. So again, let me fix the dimension to n. And we're going to have a certain volume. That's obviously an important characteristic of a region. And I'm going to assume that this is finite.

And what's the other thing that we basically want? Well, when I pick a constellation like this, by intersecting a lattice with a region, eventually I'm going to let the points in this constellation be equiprobable, as I always do in digital communications. 16 points. Say they all have probably 1/16, the uniform probability over the discrete points.

One of the approximations I am going to make -- so what's the average power of that constellation? We could figure it out by taking 1/16 times the energy. I should say the average energy, I'm sorry. We're not talking power here. We're talking energy, but I read it as P. What's the average energy? It's 1/16 times the L2 norm of each of these points, if you like that. It's simply the Euclidean squared norm.

And we get some average. But here's the key approximation principle. I'm going to say the average power of this constellation, especially as the constellations get large, is simply going to be approximated as the average energy of a uniform distribution over this bounding region. Suppose I simply put a uniform continuous distribution over this circle, this sphere, two sphere that I've used to be my cookie cutter. I'm going to say that's going to be a good approximation to the average energy of the 16 discrete points, which are, by construction, they're spread uniformly over this region.

The approximation is, I'm kind of approximating a unit impulse of weight there by distributed weight across its-- well, actually across its whole little region. If we have a hexagon around each one, uniform distribution over the hexagon. And if that's a reasonable approximation, then it's reasonable to say that the average power of the whole constellation is just the average power of a uniform distribution over this region.

It's good exercise to try it for some moderate size cases. And you'll find it is a very good approximation. In any case, it's the one we're going to use. So the other key parameter is going to be P(R), which is, in words, the average energy of uniform distribution over R. Sorry, script R. Now, when we write this out, this actually gets a little formidable. So I think it's better to remember what we're actually trying to compute. Oh, and one last thing, we're going to normalize it per dimension.

So what do I mean? What's a uniform distribution over R? This is P(x equals 1) over the volume of R for x in R and 0 otherwise. Uniform within, 0 outside. And what does it have to equal? It has to be equal to 1 over V(R), because its integral has to be equal to 1 if it's a probability distribution.

So to compute P(R), we basically take the integral from over R, of this probability distribution, 1 over V(R), times what? The energy per dimension. The total energy of x is x-squared, the L2 norm of x. Normalize per dimension -- we're going to put an n over there -- dx. So that's an equation for what it is I wanted to compute.

I think it's much harder to remember the terms in this than to remember this verbal description of what we want. But you may be more literate than I. Now, let's define -- we want to normalize quantity comparable to Hermite or coding gain over here. The normalized quantity we're going to use -- we want to take P(R) -- and what should we normalize it by to make it scale-invariant, let's say again?

Again, if I take the region, I scale the region by alpha and get alpha_R. What's going to happen to the average energy of a uniform distribution over R?

AUDIENCE: [INAUDIBLE].

PROFESSOR: It's going to go up as alpha squared. Energy scarce, goes as the square of the scale factor. So again, I want to normalize it by V(R) over 2 to the n, because this is also something that will go up as alpha squared if I scale R by alpha.

So this is what's known as the dimension-less or normalized-- it has both names-- second moment. Another name for this is the second moment per dimension of a uniform distribution over R. This is a normalized or dimension-less second moment. I think I call it normalized in the notes, but it's always called G in the literature.

And let me just introduce here-- well, what's the normalized second moment of an interval? Say, a symmetric interval around the origin. G to the minus 1 to the 1, which is like the interval that we used to pull out the m-PAM signal structure. So let's take n to be equal to 1, R just to be the interval from -1 to +1. And V(R) equals what? It's the length of R, which is 2.

What's P(R)? The integral from -1 to 1, x squared, dx, which is 2/3. I think so.

AUDIENCE: [INAUDIBLE].

PROFESSOR: Right, 1/2. Thank you. That's 1/3. All right, so what is G(R) going to be, or G going to be? This is going to be P(R) over V(R)-squared. And so this is going to be 1/3 over 4, or 1/12. OK, that's not as nice as this over here, but it is what it is.

There's a number that shows up in quantization, for instance, uniform scalar quantizer, you'll find a 1/12 in there. Why? It's because of this equation. If we ask again about the invariances of this-- this is the last homework problem for this week-- it's invariant not only to scaling, but, again, it's invariant to orthogonal transformations, because orthogonal transformations won't affect either of these things. A rigid rotation, about the origin, won't affect V(R) or P(R). And furthermore, it's invariant to Cartesian products, again.

If we just have a series of regions, a region made up basically as the Cartesian product, like an n-cube is a Cartesian product. We just pick each of the dimensions independently from some set R. Then that's not going to affect the normalized second moment either. Exercise for the student.

So from that result, I can say that G([-1 1]) to the m. Which is what? This is an m-cube of side 2. If I say each of the coordinates, x1, x2, x3, up to xm, has got to be xk n times [-1 1], then what I'm going to get, the region defined by that is an m-cubed, centered on the origin of side 2.

So the normalized second moment of that is going to be 1/12. This is where the shortcut begins to help you. I guess I'm trying to put too much on one board. But the last thing I want to put up on here is the shaping gain. The shape gain of a region is defined as the shaping gain of the region is equal to 1/12. So we're kind of taking as a baseline, the m-cube, in any number of dimensions n, over G(the region).

In other words, how much less is the dimension-less second moment, the normalized second moment, than what you would get for an m-cube? In every dimension except for one, a sphere is going to be a better region, from a second moment point of view, than an m-cube. An m-cube has corners. Really, it's pretty obvious that, if you want to minimize the normalized second moment in any number of dimensions, you would choose an m-sphere, because if you have any wrinkle that comes out of an m-sphere, anything that's not an m-sphere, you should try to squash it back in it. And that'll make it an m-sphere.

You'll take higher energy and make it lower energy for the same little unit of Play-Doh or mass stuff. So a sphere will give you the best normalized second moment in any number of dimensions n, so you're always going to get a G(R) that's less than 1/12. That's good. And this measures the amount of gain.

Again, you can measure this in dB. That's going to translate directly to dB. I suppose I could do an example for the sphere in two dimensions, but I won't. How are we doing on time? Let me see if I can do the union-bound estimate. That would be a trick, but I think it's doable, because this is where we're going.

So the union-bound estimate, which is something we did way back in chapter five or something, is basically, given a constellation, which we're going to have a lattice constellation based on some lattice lambda and some region. So this is like our m-by-m QAM, or it might be this guy, which is better than 16-QAM, in some sense. In two senses, actually.

The union-bound estimate, what is it? The probability of error is approximately equal to-- let me do it per block. So I'll have the number of nearest neighbors per block of our constellation times Q to the square root of d_min-squared of our constellation, over what is it? 4 sigma squared? 4 sigma squared.

And if you remember how we got this, we took the union-bound, we took all the pairwise error probabilities, and from that, we get contributions from kd at every kd contributions at distance d. We added them all up in the union-bound. We said, well, for moderate complexity and performance, at least this is going to be dominated by the minimum distance. So this is the minimum distance terms. This is the number of minimum distance terms. And this is error probability -- the pairwise error probability we get for each minimum distance term.

Now, to compute this, I'm going to use three approximations, but two important ones, this continuous approximation that I've already referred to. First, I'm going to say that the average power of my constellation is approximately equal to just the second moment of R. The average energy per dimension of R.

And then I'm going to say, furthermore, how many points are there in the constellation? This will affect what rate I can go at. The average number of points in the constellation -- also mention this-- is going to be approximately equal to simply the volume of my bounding region, V(R), over the volume taken up by each point in my lattice.

That makes sense, doesn't it? What I want to do here to get 16 points is I take a sphere whose area is roughly 16 times the area of one of these little hexagons or one of these little parallelograms. And I'll get about 16 points, because that's the density of it.

And there's a final little approximation, which is that I'm going to assume that we're somewhere in the interior of the lattice, so that the average number of nearest neighbors in the constellation is simply equal to the number of nearest neighbors in the lattice. So that's a reasonable approximation, and that's a very obvious one. If I take any constellation based on the hexagonal lattice, for a large enough constellation, the average number of nearest neighbors is going to be 6. If I take it on the square lattice, the average number of nearest neighbors is going to be 4.

But we know we're not terribly interested in that either. Let me do a little manipulation here. I'm going to write this as Q of d_min-squared of c, which clearly I'm going to also assume that d_min-squared of c is equal to d_min-squared of the lattice. So I'll make that d_min-squared of the lattice. And I want to normalize that by v of lambda, to the 2/n, whatever dimension n I'm in, to get something that we're familiar with. So I need a V(n) over 2 to the n.

And anticipating a little, I'm going to put a V(R) over 2 to the n. And then over here, I'm going to put V(R) over-- Is this going to come out right?-- over 2 to the n, over-- this seems upside-down. Ah, no, it isn't upside-down. Because I'm using this expression here, I do want 1 over G(R).

So this is 1 over G(R). If I multiply this by 1/12, now I get 1 over G(R), so I need to put a 1/12 here. And then I have a P(R) over 4 sigma squared. So does everyone agree I can write it that way? This'll be good. Go out with a bang.

So what are all these things? This is the coding gain of lambda. This, according to this, is the size of the constellation to the 2 over n, approximately. The rate of the constellation is log 2 times the size of lambda, R. So the rate is equal to just log 2 of this quantity. So I can say 2 to the R. This is 2 to the R equals this. So this would be 2 to the 2 over n, 2R over n. Is that right? Again, I'm unsure if that's what I want.

This is the shaping gain of the region, these three things together, times 3, times-- what's P(R) over sigma squared? That's the SNR. That's the average power per dimension. And this is the average noise power per dimension. I've used up these in that. So I've got a 1 over 1/12. That's 12 times that. So this is 3 times SNR.

And let's see. This is 2R over n is just 2 to the--

AUDIENCE: Isn't that just 2R? Because [UNINTELLIGIBLE] normalized by the dimension.

PROFESSOR: Yeah. And furthermore, it's minus, because it's upside-down. So it's 2 to the - 2R. Actually, what I want is the spectral efficiency. And the spectral efficiency is the rate per two dimensions. So this is just 2 to the minus rho. So with a little help for my friends, what's SNR over 2 to the rho?

AUDIENCE: [INAUDIBLE].

PROFESSOR: It's approximately SNR norm. Certainly true as row becomes large. It's actually SNR over 2 to the rho minus 1. So this is SNR norm. OK, so with a little hand waving and a few high SNR approximations, we get that, in summary, for the UBE, the probability of error is approximately K_min(lambda) times Q to the square root of the coding gain of the lattice, times the shaping gain of the region, times 3SNR norm. And that's a wonderful approximation.

Let me just call out some of its properties to you, and then we'll talk about them next time. But first of all, there's nothing here-- all these terms involve either lambda or R. So we have completely separated, in this analysis, the effects of the region from the effects of the lattice. We can choose the lattice and the region independently. And they independently affect how good a performance we get. For the baseline, which was either m-PAM or m-by-m QAM, we have, for the baseline PAM or QAM, we've set it up so that both the coding gain of the lattice and the shaping gain of the region are equal to 1.

So for the baseline, this gives us what we already developed back in the early stages. I won't worry about this -- Q-hat, but it does give you the right coefficient out here, too, depending on whether you normalize it per two dimensions or what. It just gives you Q to the square root of 3SNR norm, which I hope you remember from chapter five. So this is our baseline curve for m-PAM or m-by-m QAM.

We just plotted versus SNR norm, the probability of error per two dimensions, which is what I recommended. And we got some curve. So this is the baseline. And now we've got everything separated out in a nice way, so that if somebody gives us a lattice with a certain coding gain and a region with a certain shaping gain, those both just add up as gains.

If that's all we had, forget the coefficient, we would just move to the right here, and it would give us effective reductions and the required SNR norm by the coding gain of the lattice and the shaping gain of the region. And we'd get the same curve moved out over here.

Again, as in the binary case, typically we're going to get a larger number of near neighbors, leading to a larger coefficient out here. We want to normalize this per two dimensions. Per two dimensions, it's only 4 for the baseline. But in general, we're going to get a KS of lambda, the number of nearest neighbors per two dimensions, which is going to raise this a little. And that will lower the effective coding gain down here.

But we've basically, by these large constellation, high SNR approximations, we've really reduced the analysis of lattice constellations to a very simple task, just to analyze what's the coding gain, what's the shaping gain, number of nearest neighbors, plot the performance curve, end of story. So we'll start there next time.