L8: StreamIt Language

Related Resources

Lecture Notes (PDF - 1.1 MB)

• Download English-US transcript (PDF)

The following content is provided under a Creative Commons license. Your support will help MIT OpenCourseWare continue to offer high quality educational resource for free. To make a donation you or view additional materials from hundreds of MIT courses, visit mitopencourseware@ocw.mit.edu.

PROFESSOR: So let's get started with the second lecture for today. So I guess one thing multicores did, is really shatter this nice view of writing in your programs and hardwares to take care of, giving you performance. So hardware just kind of completely gave that up. But so what you're doing in this class, is you're trying to do it by yourself. Give all the responsibility back to the program. And you realize as you go, it's a much harder job. I mean, this is not simple programming. So you need to have, you don't have MIT class students on every company to do this, so we need to have some kind of middle ground. And so some of the stuff we have been doing is trying to figure out are there any middle ground. Can you actually take some of that load away from the user into things like languages and compilers. So we will talk about some of those. So right now we are kind of switching from directly doing what's necessary, to do the Cell project into going breadth So this lecture, and then we will sit back and do a little bit of debugging and performance work and that will be directly helpful. And then next week we'll have lots of guest lectures to kind of give you breadth in there. So you'll understand no just Cell programming but parallel programming and parallel processing, what the world is like beyond that. So today we're going to have Bill talk about streams.

BILL THIES: OK very good. So my name is Bill Thies. I'm a graduate student working with Saman and Roderick, and others here. And I'll talk about the StreamIt language. So why do we need a new programming language? Well we think that languages haven't kept up with the architectures. So one way to look at this is that if you look back at previous languages, look at C with von-Neumann machine. Now I grew up in rural Pennsylvania not too far from Amish Country. And so to me these go together just like a horse and buggy. OK they're perfectly made for each other. They basically go at the same rate. Everything is fine. But the problem is, in comes the modern architecture. OK this is an F-16. you have a lot more that you can do with it then, then with the horse and buggy.

So how do you program these new architectures? Well architecture makers these days are basically faced with a really hard choice. On the one hand, you could get a really cool architecture and develop an ad hoc programming technique where you're really just leaving it to the programmer to do something complicated to get performance. And unfortunately I think that's the route that they took. I mean fortunately for the industry, but unfortunately for you, I think that's the route they took with cell which means all of you are going to become basically fighter pilots. You have to learn how to fly the plane. You have to become an expert. You're going to become the best people at programming these architectures. And unfortunately the only other option is to really bend over backwards to support the previous era of languages like C and C . And you can see what's coming here, it's just hard to get off the runway. So you don't want this situation. Out of consideration for whoever is in the buggy, hopefully you'll never take off.

So looking from a more academic perspective, why do we need a new language right now? So if you look back over the past 30 years, I know you've seen this graph before. We were dealing with just one core in the machine for all this time. And now we have this plethora of multicores coming across the board. So how did we program these old machines? Well we had languages like C and FORTRAN that really have a lot of nice properties across these architectures. So it was portable, high-performance, composable-- you could have really good software development-- malleable, maintainable, all the nice things you'd like to see from a software engineering perspective. And really if you wrote a program back in 1970, you could keep it in C and have it continue to leverage all the new properties of machines over the past 30 years. So just one fine out of the box. And looking forward, that's not going to be true. So for example, we could say that C was the common machine language. That's what we say for the past 30 years, was common across all the machines. But now looking forward, that's not going to be true anymore. Because you have to program every core separately.

So what's the common machine language for multicores? We really think you need something where you can write a program once today, and have it scale for the next 30 years without having to modify the program. So what kind of language do you need to get that kind of performance? Well let's look a little deeper into this notion of a common machine language. So why did it work so well for the past 30 years? Well on uniprocessors, things like C and FORTRAN fran really encapsulated the common properties. So things like a single flow of control in the machine, a single memory image, are both properties of the language. But they also hid certain properties from the programmer. So they hid the things that were different between one machine and another. So for example, the register file, the ISA, the functional units and so on. These things could change from one architecture to another. And you didn't have to change your program because those aspects weren't in the programming language. So that's why these languages were succeeding.

And what do we need to succeed in the multicore era from a language perspective? Well you need to encapsulate the common properties again. And this time it's multiple flows of control that you have for all the different cores, and multiple local memories. There's no more monolithic memory anymore, that everyone can read and write to. Also you need to hide some of the differences between the machines. So some cores have different capabilities. On cell there's a heterogeneous system between the STEs and the PPE. Different communication models on different architectures, different synchronization models. So whatever common machine language we come up with, we'll have to keep these things hidden from the programmer.

Now a lot of different researchers are taking different tacts for how you want to invent the next common machine language. And the thrust that we're really excited about is this notion of streaming. So what is a stream program? Well if you look at a lot of the high-performance systems today-- including Powerpoint which is running this awesome animation-- you can basically see that they're based around some stream of data. So audio, video, like HDTV, video editing, graphic stuff. I think actually, a lot of the projects in this class that I looked at, would fit into the streaming mold. Things like the software radio, array tracing, I probably don't remember them all. But when I looked at them, they all looked like they had a streaming component somewhere in there.

So what's special about a stream program compared to just a normal program? Well they have a lot of attractive properties. If you look at their structure, you can usually see that the computation pattern remains relatively constant across the lifetime of the program. So they have some well-defined units that are communicating with each other. And they continue that pattern of communication throughout. And this really exposes a lot of opportunities for the compiler to do some optimizations that it couldn't do on just an arbitrary general purpose program. And if you saw before, we have basically all the types of parallelism are really exposed in a stream program. There's the pipeline parallelism between different producers and consumers. There's the task parallelism basically going from left to right. And also data parallelism which means that a single one of these stages can sometimes be split to apply to multiple elements in the data stream.

So when you're thinking about stream programming, there's a lot of different ways you can actually represent the program. So whenever you have a programming model, you have to answer these kinds of questions. For example do the senders and the receivers block when they try to communicate? How much buffering is allowed? Is the computation deterministic? What kind of model do you have in there? Can you avoid deadlock? Questions like these, and we could spend a whole lecture answering these questions, putting them in different categories. But what I want to just to do, just to give you a feel is touch on kind of three of the major models that you might see come up in different kinds of programming models. And I'll just touch Kahn process networks, synchronous dataflow, and communicating sequential processes, or CSP.

So just one slide on these models. So let's compare them a little bit. First there's the Kahn process networks. So this is kind of the simplest model. It's very intuitive. You just have different processes that are communicating over FIFOs. And the FIFO size is conceptually unbounded. So to a first approximation, it's kind of like a Unix pipe. These processes can just read from the input, and they can push onto their outputs without blocking. But if they try to read from an input they do block until an input is available. And the interesting thing is that the communication pattern can actually be dependent on the data. So for example I could pop an index off of one channel, and then use that index to determine which other channel I'll read from on the next time time step. But at the same time it is deterministic. So for a given series of input values on the stream, I'll always have the same communication pattern that I'm trying from the other input. So if it's a deterministic model, that's a nice property. Let's see, what else to say here? There's actually a few recent ventures that are using Kahn process networks. So there's commercial interest. For example Ambric is a startup that I think will be based on a Kahn process network for the programming model.

Looking at another model called synchronous dataflow, this is actually what we use in the StreamIt system. And compared to Kahn process networks, it's kind of a subset. It's a little bit more restrictive. So if you look at the space of all possible program behaviors, Kahn process networks are a pretty big piece of the space. And then synchronous dataflow is kind of a subset of that space where you know more about the communication pattern at compile time. So for example, in synchronous dataflow, the programmer actually declares how many items it will consume from each of its in put channels on a given execution step. So there's no more data dependence regarding the communication pattern. It'll always input some items from some of the channels and produce some number of items to other channels. And this is a really nice properties because it lets the compiler do to scheduling for you. So the compiling can see who's communicating to who and exactly what pattern. And it can statically interleave the filters to guarantee that everyone has enough data to complete their computation. So there's a lot of interesting optimizations you can do here. That's why it's very attractive for StreamIt. And you can statically guarantee freedom from deadlock, which is a nice property to have.

The last one I want to touch on is communicating sequential processes or CSP. And in the space of program behaviors, it's kind of an overlapping that from Kahn processing networks, and adds a few new semantic behaviors. So the buffering model is basically rendezvous communication now. So there's no bothering in the system. Basically anytime you send a value to another process, you have to block and wait until that process will actually receive that value from you. So everyone is rendevouzing at every communication step. In addition to that, they have some sophisticated synchronization primitives. So you can for example, discuss alternative behaviors that you have. You can either one thing or another which will introduce the nondeterminism in the model. Which could be a good or a bad thing depending on the program you're trying to express. And pretty much the most well-known encapsulation of CSP is this occam programming language invented quite a while ago. And some people are still using that today. Any questions on the model computations? OK.

So now let me get into what StreamIt is. So StreamIt is a great language. It's a high-level architecture-independent language. Oh question the back.

AUDIENCE: With the CSP I'm trying to understand exactly what that means or how it's different. Is basically what's it's saying is you have a bunch processes and they can send messages to each other.

BILL THIES: So all these models have that property.

AUDIENCE: They all fit into that. But it seems like from your explanation of CSP, that that was just sort of the essence of CSP is it more specific?

BILL THIES: So CSP is usually associated with rendezvous communication. That's the side of programs that fit inside Kahn process networks. It's any communicating model where you basically have no buffering between the processes. Now the piece that sits outside is usually lumped with CSP, or especially with occam They have a set a primitives that are richer in terms of synchronization. So for example, you can have guards on your communication. Don't execute this consumption from this channel until I see a certain value. So there's some more rich semantics there. And so that's the things that are usually outside. They're outside the other models. Does that make sense? Other questions?

OK so StreamIt. OK so StreamIt is architecture-independent. It's basically a really nice syntactic model for interfacing with these lower level models of computation for streaming. And really we have two goals in the StreamIt project. And the first is from the programmer's side. So we want to improve the programmer's life when you're writing a parallel program. We want to make it easier for you to write a parallel program then you would have to do in C or a language like Java, or any other language that you know. And at the same time, we want scalable and portable performance across the multicores. So an interesting thing these days is you'll find, is it's often very hard a tempt the programmer to switch to your favorite language based solely on performance. Or at least this has been the story in the past. It may change, looking forward. Because it's a lot harder to get performance these days. But usually you have to offer them some other carrot to get them on board. And you know the carrot here is that it's really nice to program in. It's fun to program in. It's beautiful. It's a lot easier to program and stream it then it would be in something like C or Java for a certain class of programs. So that's how we get them on board, and then we also provide the performance. We're mostly based on the synchronous in dataflow model. In that when there are static communication patterns, we leverage that from the compiler side. So I'll also tell you about some dynamic extensions that we have, that is the much richer model of communication.

So what have we been doing in the Streamit Project? We have kind of a dual thrust within our group building on this language. So the first thrust is from the programmability side, looking at applications and programmability What can we fit into the streaming model? And we're also really pushing the optimizations. So what can you do from both a domain specific optimization standpoint, as kind of emulating a DSP engineer or a signal processing expert in the design flow. And also architecture specific optimizations. So we've been compiling for a lot of parallel machines. And we were hoping we could have a full system for you guys, this IEP, so you could write it then stream it and then hit the button. And it would work the whole way down to cell. Unfortunately we're not quite there yet. But we do have a pretty robust compiler infrastructure. And you can download this off the web and play with it if you want to.

One of our backends that we've released so far actually does go to a cluster of workstations. So it's kind of an MPI-like version of C. It uses Pthreads for the parallelism model. And I mean, depending on what kind of a hacker you are, you actually might be able to lower that down onto cell. So some of the stuff you might be able to use if you're have some initiative in there. And of course we'd be willing to work with you on this as well. so we have lots optimizations in the tool flow. And actually Saman will spend another lecture focusing on the StreamIt compiler, and how we get performance out of the model.

OK, so let's just jump right in and do the analog of Hello World in StreamIt. I'm going to kind of walk you through the language and show you the interesting pieces from an intellectual standpoint. What's interesting about a streaming model that you can take away. So instead of Hello World, we have a counter. Since we're dealing with stream programs here, you're not usually doing text processing. So how do you write counter? Well there are two pieces to the program. The first is kind of the interconnect between the different components. That's what we have up here. We're saying the program is a pipeline with two stages, it has a source, and it has a printer. And then we can write the source and the printer as filters. We call those basic building blocks filters in Streamit. So the source will just have a variable x that it initializes is zero. And then we have a work function which is automatically called by our runtime system every time through the steady state. So this work function well push one item on to the output channel, and it'll increment the value afterward. Whereas the intPrinter at the bottom here, will input one value. And its work function here just pops that value off the input tape, and prints it to the output.

Now how do we run this thing? Well there's no main function here like you see in Hello World. Is there comment? Oh, sorry.

AUDIENCE: The two meanings of push and two meanings of pop.

BILL THIES: Two meanings?

AUDIENCE: Push 1, 2, 3.

BILL THIES: Yeah, yeah. So the first push here is just declaring that this work function will push one item to the output tape. So this is the synchronous dataflow aspect. Were associating an output rate with this work function. So that's a declaration here. Whereas this push is just actually executing the push onto the output.

So how do we run this thing? Well we compile it with a StreamIt compiler, store C into a binary. And then when we run, we run for a given number of iterations. So you don't just call it once. Our model is that this is a continuous stream of data going through the program. And so when you run it, you run it for some number of iterations, or basically input or output items, is what you're running it for. So if you run this for four iterations, it would print in the 033. So we can leverage this steady flow of data. Yeah Amir?

AUDIENCE: 1, 2, 3, 4, pushing X plus plus .

BILL THIES: I think the plus, plus is executed after the push.

AUDIENCE: Push plus plus?

BILL THIES: So it starts at zero. So I think a PostFix expression executes after the actual obsession. Yeah. Yeah. Other questions? OK so let's step up a level and look at what we have in StreamIt. So the first question is, how do you represent this connectivity between different building blocks? How do you represent streams? And if you look at traditional programming models, kind of the conventional wisdom is that a stream program is a graph. You have different nodes that are communicating to each other. And graphs are actually kind of hard to analyze. They're hard to represent. They're a little but confusing. So the approach we decided to take in StreamIt is one of a structured computation graph. So instead of having arbitrary inner connections between the stages, we have a higher hierarchical description in which every individual stage has a single input and a single output. And you can compose these together into higher level stages. Of course there's some pages that do split and join with multiple inputs. We'll get to that.

So the analog here is kind of ah analogous to structured control flow, in your favorite imperative programming language. Of course there was a day when everyone used goto statements instead of having structure control flow. We've got a fan of goto statements in the audience? OK, I'll get to you later. But the problem was, it's really hard to understand the program that's jumping all over the place because there's no local reasoning you can have. You know you're jumping to this location, you're coming back a different way. It's hard to reason about program components. So when people went to structured control flow, there's just if else, four loop statements. Those are the two basic constructs. You can basically express all kinds of computation in those simple primitives. And things got a lot simpler. And you know people objected at one point even. You know what about a finite-state machine? Don't you need goto statements for a finite-state machine, going from one state to another another. And now everyone writes in FSM with a really simple idiom. You usually have a while loop around a case statement. Right, you have a dispatch loop. So and now whenever you see that pattern you can recognize, oh there's a finite-state machine. It's not just at set of gotos. It's a finite-state machine.

So we think there are similar idioms in the streaming domain. And that's kind of the direction we're pushing from a design standpoint. So what are our structures that we have? Well here are our structured streams. At the base we have a filter. That's just the programmable unit like I showed you. We have a pipeline, which just connects one stream to another in a sequence. So this gives you pipeline parallelism. There's a splitjoin where you have explicit parallelism in the stream. So I'll talk about what these splitters and joiners can do. It's basically a predefined pattern of scattering data to some child streams, and then gathering that data back into a single stream. So the whole construct still remains single input and single output.

Likewise a feedback loop is just a simple way to put a loop in your stream. And of course these are hierarchical. So all of these green boxes can be any of the three constructs. So that's how you can have these hierarchical graphs. And again, since everything is single-input single-output, you can really mix and match. You know choose your favorite components, and they'll always fit together. You don't need to stitch multiple connections.

So let's dive inside one of these filters now. And I gave you a feel for how they look before, but here's a little more detail. So how do we program the filter? Well a filter just transforms one stream into another stream. And here it's transforming a stream of floating-point numbers into another floating-point number stream. I can take some parameters at the top. These actually fixed at compile time in our model, which allows the compiler to really specialize the filters code depending on the context in which it's being used. So for example here, we're inputting N in a frequency. And then we have two stages of execution. At initialization time-- this runs one at the beginning of the whole program-- we can calculate some weights for example, from the frequency. And we can store those weights as a local variable. So you can think of this kind of like a Java class. You can have some member variables. You can retains state from one execution to the next.

The work function is the closest thing to the main function. This is called repeatedly in the steady state. And here are the IO rates of the work function. This filter actually peaks at some data items. That means that it inspects more items on the input channel than it actually consumes on every iteration. So we'll look at N input items, and we'll push one new item onto the output and pop one item from the input tape every time we execute. So here we have a sliding window computation. It means the next time through, we'll just slide this window up by one and inspect the next N items on the input tape. And inside the work function you can have pretty much general purpose code. Right now we just allow pointers and a few other things to keep things simple. But the idea is this is general purpose imperative code inside the work function.

Now what's nice about this representations of a filter is for one thing is this peak function. So what we really have is a nice representation of the data pattern, that you're reading the data on the input channel. And if you look at this for example in a language like C, it's a lot messier. So usually when you're doing buffer management, you have to do some modulo operations. You have to keep a circular buffer of your live data. And increase you know, a head or tail pointer and mod around the side with some kind of modulo operation. And for a compiler, this is a real nightmare. Because modulo operations are kind of the the worst thing to analyze. You can't see what it's actually trying to read. And if you want to map this buffer to a network or to a combined communication with some other actor or filter in the graph, it's pretty much impossible. So here is we're exposing that all to the compiler. And you'll see how that can make a difference. And also it's just a lot easier to program. I mean, I don't like looking at the code. So I'm going to go to the next slide.

OK, so how do we piece things together? Let's just build some higher level components. So here's a pipeline of two components. And we already saw a pipeline. You can just add one component after another. And add just basically has the effect of making a queue, and just queueing up all of the components that you added, and connecting them one after another. So here we have a BandPastFilter by connecting a LowPassFilter and feeding it's output into a HighPassFilter. You end up with a BandPassFilter.

OK what about a splitjoin? How do we make those? So a splitjoin has an add statement as well. And here we're adding components in a loop. So what this means is now when we say add, we're actually adding from left to right. So instead of going top to down, we're adding from left to right across the splitjoin. And we can actually do that in a loop. So here we're intPrinting a parameter N. And depending on that value, we'll add N BandPassFilters to this splitjoin. So it's kind of cool, right? Because you can input a parameter, and that parameter actually affects the structure of your graph. So this graph is unrolled at compiled time by our compiler, constructing a big sequence of computations. And it can resolve the structure and communication pattern in that graph, and then map it to the underlying substrate when we compile.

Also to notice here are the splitter and the joiner. So we have a predefined set of splitter and joiners. I'll go into more detail later. But here we're just duplicating the data to every one of these parallel components, and then doing a round-robin join pattern where we bring them back together into a single output stream. And if you want to do an equalizer, you basically need an adder at the bottom to add the different components together. And another thing you can notice here is that we have some inlining going on. So we actually embedded this splitjoin inside a higher level pipeline.

So what this does is it prevents you from having to name every component of your stream. You can have a single stream definition with lots of nested components. And the natural extension is, you can basically scale-up to basically the natural size of a procedure, just like you would in an imperative language. And here is for example, an FM radio where we have a few inline components. And the interesting thing here is that there's a pretty good correspondence between the lines of the text and the actual structure of the graph. And that's something that's hard to find in an imperative language. I mean if you want to for example, stitch nodes together with edges, it's often very hard to visualize the resulting structure of the graph that you have. But here if you just go through the program, you can see that the AtoD component goes right over to the AtoD, the demodulator to the demodulator, and so on. And even for the parallel components, you can kind of piece them together.

so that's kind of how we think of building programs. Any questions so far? OK so this is kind of how you go about programming in StreamIt. But programming is kind of a chug n' plug activity right? Nobody wants to be a code monkey. The reason we're all here is to see what's beautiful about this programming model. Right? Don Knuth said this, a famous computer scientist from Stanford during his Turing Award Lecture you know,

"Some programs are elegant, some are exquisite, some are sparkling. My claim is that it is possible to write grand programs, noble programs, truly magnificent ones!"

Right. We want the best programs possible. It's not just about making it work. We want really beautiful programs. So what's beautiful about the streaming domain? What can you go away with and say wow, that was a really beautiful expression of the computation. Well for me I think one of the interesting things here is the splitjoin contruct. Splitjoins can really be beautiful. You know some mornings I just wake up and I'm like, oh I'm so glad I live in a world with splitjoins. You know? And and now splitjoins will be part of your world. You can say this tomorrow. This is just wonderful.

So OK, what do we having in the splitjoin constructs? You can duplicate data. You can do a round-robin communication pattern from one to another, or round-robin join. Now the duplicate is pretty simple. You just take the input stream and duplicate it to all the children. No problem. What do you do for a round-robin? Well you path N items from the input to a given child. So for example, if N is 1, we'll just distribute one at a time to the child streams. And you get a pattern like this, a round-robin just going across. And you can do the same thing on the joiner side. Let's say you're joining with a factor of one. You're just reading from the children and putting them into a single stream. OK so the pretty colorful, but nothing too fancy yet.

Let's consider a different round-robin factor. So a round-robin of 2 means that we peel off 2 items from the input, and pass those items to the first output. OK there actually is going to be a quiz on this. So ask questions of this doesn't make sense. OK pass the next 2 items, and 2 items round-robin like that. And you can actually have nonuniform weights if you want to. So on the right let's say we're doing round-robin 1, 2, 3. That means we pass 1 item from the first stream, 2 items from the next stream, and then 3 items from the third stream. OK, pretty simple. We're just doing 1, 2, and 3, and so on. Does that make sense? I'm going to build on this so any questions? OK this was colorful but this totally beautiful yet. So what's beautiful about this?

well let's see how you might write a matrix transpose. OK, something you guys have probably all written at one point in your life is transposing a matrix. Let's say this matrix has M rows, and it has N columns going across. And we're starting with a representation in which the stream is basically, I think this is row major order. The first thing that you're doing is going across the rows before you're going to the next the next row. I'm sorry you're going across the columns, and then down to the next row. So it's zigzagging like this. And you want to pass through a transpose, so you zigzag the other way. You do the first column, up, and then the next column, and so on. And this comes up a lot in a stream program.

So it turns out you can represent this as a splitjoin. Oh that's not good. Just in my moment in glory as well. OK, yeah slides. You can download slides. Actually I can do this on the board. This is the thinking part anyway. Yeah, could you just bring up GMail? I have a backup on GMail. Can we focus on the board? So this is going to be a little exercise anyway. OK, here's what we had. We had M, M rows, N columns. We started with an interleaving like this. Right, and we want to go into a splitjoin. And this will be a round-robin construct. And what I want you to do is fill in the round-robin weight, and also the number of the streams. And you can have a round-robin at the bottom. And when it comes out, you want the opposite interleaving. This is M and N.

OK so there are 3 unknowns here, what you're doing the round-robin by-- can you guys see that over there-- how many parallel streams there are, and what you're joining the round-robin by. Ok so I'm going to give you a minute to think about this. Try to think about this. See if you can figure out what these constants are. You just basically want to read from this data in a row major pattern, M rows and N columns, and end up with something that's column major. What are the values for the constant? Does it makes sense? Ask a question if it doesn't make sense. Yeah?

AUDIENCE: So we assume that values are going to, based on the line that you drew, across it, tests like a stream line?

BILL THIES: Right, right, right. So those values are coming down the stream. You have a 1-dimensional stream. It's interleaved like this. So you'll be reading them like this. And then you the output a 1-dimensional stream that is threading the columns. Does that make sense? Somebody ask another question. Yeah

AUDIENCE: So the actual matrix transpose codes, it's my understand that nobody actually does it sequentially like that because of locality issues. Instead it's broken up into blocks.

BILL THIES: So there are ways to optimize this.

AUDIENCE: And after you've sort of serialized it, can you then capture..

PROFESSOR: Guys, [UNINTELLIGIBLE PHRASE] has a blocking segment. So you can heirarchically do that. So, normally what happens is you do the blocks and inside the blocks, you can do it again. You can do it at two levels, basically.

BILL THIES: Any hypotheses? Anyone?

AUDIENCE: Is it N for first one, M for the second one?

BILL THIES: OK, what do we have

AUDIENCE: N for the first one, and M for the second?

BILL THIES: N,M and

AUDIENCE: Same for the M?

BILL THIES: Same, M?

AUDIENCE: Yeah.

BILL THIES: OK OK, this is a hypothesis. Other hypotheses?

AUDIENCE: N, M, 1.

BILL THIES: OK, anyone else? Any amendments?

AUDIENCE: How about 1, 1, 1?

BILL THIES: 1, 1, 1? OK lottery is closing. Yep?

AUDIENCE: !, M, M

BILL THIES: 1 M, M? 1, M, M, OK and last call? OK I believe two of the ones submitted are correct. Is this and, yeah? OK I think this, N, M, 1, works and 1, N, M, works. So let me explain the 1, N, M, this is how I like to think about it. One way to think about this is we just want to move the whole matrix. You doing OK, yeah? OK we just want to move the whole matrix into this splitjoin. So the way we can do that is have M columns of the splitjoin since we have N columns of the matrix. And what we'll do is we'll just do a round-robin one at a time, from the columns of the matrix into the columns of the splitjoin. So we'll take the first element, send it to the last, next element, next column, and so on. So we get the whole matrix here, Now I want to read it out column-wise. So we'll do a joiner of M. We'll read M from the left stream that'll read all M items from the columns, send it out, and then M items in the next column, and then send it out. Does that make sense? How many people understood that? All right,

So if you think about it, you can also do it in M, N, 1. That's basically, yeah, it's very similar. OK there we were. And yes. We can do 1, N, M. We basically read the matrix down, and then pull it down into column. And it's very easy to write this as a transpose. So we just have a transpose filter in which we're doing nothing. No competition in the actual rows or the actual contents of the splitjoin. And we just split the data by 1, have N different identity filters, and then join it back together by M. Any questions about this? An interesting way to write a transpose.

OK so there's one more opportunity to shine here. And that's a little more interesting permutation called a bit-reversed ordering. And so this comes up in an FFT and another algorithm. The permutation here, is that we're taking the data at the index n. And let's say n has binary digits b sub 0, b sub 1, up to b sub k. And we want to rearrange that data, so this data goes to a different index with the reversed bits of its index. So if it was and index n before, it ends up at b sub k down to b sub 1, b sub 0. So for example, let's just look at 3-digit binary numbers. If we have 0, 0, 0, this is 1 input item. We're reversing those digits, you still get 0, 0, 0. Item at index 1 will be 0, 0, 1. We want to reorder that to index 4. OK, 1, 0, 0, 0, 1, 0, stays the same, 0 1, 1, goes to 1, 1, 0 shifts over. And from there on, it's symmetric.

AUDIENCE: [UNINTELLIGIBLE PHRASE]

BILL THIES: Sorry?

AUDIENCE: [UNINTELLIGIBLE PHRASE].

BILL THIES: OK. So here I'm writing the indices. So I'm not writing the data. So index 0, 1, 2, 3 up through index 8., or index 7. OK and on the bottom you have indices 0 through 7. So the data will actually be moved, reordered like that. Does that make sense? It's a reordering of data. Does this transformation make sense? Other questions?

OK, it turns out you can write this as a splitjoin. And you just need 3 different weights for the round-robins. OK round-robin with 1 weight on the top here, and two different round-robin weights on the bottom. And here I'm assuming you have 3 binary digits in your index. So you're reordering in groups to 8. OK, so let me give you a second to think about this. What are the values for these weights? I'll give you a hint in just a second, or ask another question. Yes?

AUDIENCE: [UNINTELLIGIBLE PHRASE].

BILL THIES: So what we're doing here, is we're exposing the communication pattern. That's the thing. If you write this in an imperative way, you end up basically having your, you're conflating basically the data dependencies with the reordering pattern. So what I'm trying to convey here, is how you can use the streaming model to show how you're sending data around. Because when you're on an architecture-like cell, everything is about the data motion. You're taking data from one place and you're trying to efficiently get it to the producers or the consumers. And you really need to-- the compiler needs to understand the data motion. And also, it's just another way of writing it, which I think it's actually easier to understand once you see it. It's a way to think about the actual reordering from a theoretical standpoint

Any wagers on this? So what we to do, if you think about the bit-reverse ordering, what we want to do is distribute the data by the low-order bits. And then gather the data by the high-order bits. So you want a fine-grained parity when you're shuffling. You can also do it the other way around. It's totally symmetrical. But one way to think about it is, you want a fine-grained parity when you're distributing data, and then a course-grained when you're coming together. Anyone see it? Give you ten more seconds. Come on? Yeah it is a little bit tricky.

OK well let me explain how it works. So 1, 2, and 4. Ok so what these round-robin 1 splitters do, is these are basically the fine-grained parities. So OK, the first round-robin, that will send all the even bits to the left, and all the odd values to the right. Right, that's the lowest order bit. Because it's doing every other one, shuffling it left and right. So this round-robin is seeing only the even values. Now it's going to split them up based on who's divisible by 4. Now we'll go to the left or go to the right. This is basically shuffling in the order of the bits, from low-order bits to high-order bits. So these will be ordered in terms of their low-order bits. And now we just want to read them out from left to right. Just take the order that we made with those round-robins, and read them out from left to right. And since they have 8 values, you can do that just by chunking them up. We'll read 2, 2, and then we'll read these two, 2 and 2, and now put all 8. Does that make sense?

OK, so yes, it's a bit clever. So I think it's a nice way of thinking about what a bit-reversed ordering means. And you can write this in a very general way. You just have a recursive bit-reversed filter for N values. And base case, you only have 2. So there's no reordering to do when you when you think about it. So you're not doing any computation. Otherwise yo have a round-robin split in half, and then have a coarse-grain joiner. So, you get a structure like this. If you're, as you're building up, just distributing and then bringing back together in a course-grained way. OK. Let's see how do I don't do this? OK so one thing to notice, there's one more example of a splitjoin. Question?

AUDIENCE: [UNINTELLIGIBLE].

BILL THIES: OK so in general, at the base of this hierarchy, we could've added some other filter to do some competition. Identity just means we're doing no computation. It's a predefined filter that just does nothing.

PROFESSOR: On complex data.

BILL THIES: On complex data. Sorry this is a templated filter. So we're reordering complex values. And we're passing the input to the output. Amir?

AUDIENCE: [UNINTELLIGIBLE]

BILL THIES: In general the language does not have support for templates. We only do it for these base classes. That's more of an implementation detail. Yeah.

AUDIENCE: [UNINTELLIGIBLE].

BILL THIES: Right now there isn't, but nothing fundamental there. Yeah? Other questions? Yeah?

AUDIENCE: How did you know that there are two filters after that?

BILL THIES: Two filters, sorry here?

AUDIENCE: [UNINTELLIGIBLE].

BILL THIES: OK. OK. So we have two add statements between the split and the join. So that branches to two parallel streams. Is that your question?

AUDIENCE: Yeah. How do you that theorem, that there's not like three.

BILL THIES: So the compiler will analyze this, the compile time. And it'll know these values of N and propagate them down at compile time. So it'll basically symbolically evaluate this code, see there are two branches, and you can unroll this communication pattern.

AUDIENCE: I think another way of thinking about it is, each add statement essentially adds another branch in your splitjoin.

PROFESSOR: That is another box.

BILL THIES: It's about to get clear actually. Other questions?

AUDIENCE: [UNINTELLIGIBLE]

BILL THIES: That's one way to think about it, yeah. Wait say again, counting?

AUDIENCE: A rating sort.

BILL THIES: A rating sort, right. OK, well is it sorting? It's not really sorting.

AUDIENCE: Well it's not really sorting.

BILL THIES: It's a permutation.

AUDIENCE: Could you do a rating sort?

BILL THIES: You could do a rating sort. So actually, what I want to show next is how you can morph this program into a merge sort by changing only a few lines. OK look carefully. Don't blink. OK there's merge sort. So very similar pattern. This is one of those idioms. It's a recursive idiom with splitjoins. But now in the base case, we have a sort. So we would basically branch down. What we ended up with was a sort in the base case. We're just sorting a few values. And we call [? merge sort ?] twice again, and then we do a merge. So instead of identity at the base case here, we now have a basic sorting routine. And we merge those results from both sides. And the only thing I changed in terms of the communication rate, is to be more efficient we just distributed data in chuncks instead of doing a fine-grained splitting. Actually you do it however you want in a merge sort. But this is chunked up. And let's just zoom in here. This is how a merger sort looks in StreamIt. So we split the data two ways, both directions, come together, do a sort on both sides, and then you merge.

And so by having the, you know you can interleave pipelines and splitjoins like this. So you have these hierarchical structures that are coming back together. Does this make sense? OK. I'm going to hold off on messaging actually. Let me see, what do I want to cover? OK let me actually skip to the end. Oh, I can show you this. Yeah, I'm going to cut short a little bit. So here's how other programs look written in StreamIt. OK, you can have a Bitonic sort. OK so you see a lot of these regular structures. And the compiler can unroll this and then match it to the substrate. This is how and FFT looks. It's quite an ah elegant implementation of an FFT. It'd be good to go into in more detail.

You can do things like block matrix multiply. You don't always have to have column or row-wise ordering. It's natural to split things up like this. We have a lot of DSP algorithms, the filter bank, FM radio with equalizer, radar array front end. Here's an MP3 decoder. And let's see, I'm going to skip this section and just give you a taste for the end here. I'm skipping a hundred slides. Yeah.

OK so if I give you a feel. Our biggest program written in StreamIt so far, is the complete MPEG-2 encoder and decoder. So here is MPEG-2 decoder. And I think you've seen block diagrams of this already in the class. And so it's a pretty natural expression. You can really get a feel for the high-level structure of the algorithm mapping down. And for example, here on the top we're doing the spatial decoding looking inside each frame. Whereas at the bottom we're doing the temporal decoding between two frames, the motion compensation. And one thing that I didn't have a chance to mention, is that we have a concept of teleport messaging. What this means is, I showed you how the steady state flow data goes between these actors in the stream. But sometimes you want to control the stream as well.

For example, this is a variable length decoder at the top. It's parsing the input data. It might want to change how the processing is happening downstream. For example, say that you have-- you know in this case you have different picture types coming in. And you want to tell other components to change their processing based on a non-local effect. And that's hard to do if you just want static data rates. But what we have is this limited notion of limited dynamism, where you're basically poking into somebody else's stream. And we let you do that very precisely. I don't have time to go into the details, but you can basically synchronize the arrival of these messages with the data that's also flowing through the stream, And so in this case, were sending through the picture type. And it really simplifies the program code. I didn't have time for details, but why don't we put in the slides anyway, if you're interested.

And if you do a similar communication pattern in C, it's a little bit of a nightmare. You have all these different, basically memory spaces, different files. And the control information is basically going left and right all over. So this really helps both the compiler and the programmer as well in StreamIt.

So it's all implemented. It's about 2,000 lines of code in StreamIt. Which is about 2/3 of the size of the C code, taking into account similar functionality there. And it's a pretty big program. You can write 48 static streams. And then we expand that to more than 600 instantiated filters. So this gives you a lot of flexibility when you're trying to get parallelism. Question?

AUDIENCE: When a compiler downloads all bytes?

BILL THIES: Oh the object code, you mean. OK, so right now our current implementation, we duplicate a lot of code. So it end up being bigger than it needs to be. There's no reason for us to do that. That's kind of a-- we have a research compiler that make that easy.

AUDIENCE: So object-wise , its not data.

BILL THIES: Object-wise we still need to do that comparison. Yeah that's a good question. Yeah. Other questions? OK so let me cut to the end. OK, so we have the StreamIt language. And we think it really preserves the program structure. It's a new way of thinking about how you orchestrate the data reordering with the splitjoins, showing you who is communicating to who, and how you can stitch together different pieces in your program development. And again, really our goal is to get this scalable multicore performance. But you can't get people on board just on a performance stat. You need to show them a new programming model that actually makes their lives easier. So that's what we're working on. And thinks with listening.

[APPLAUSE]

BILL THIES: Any last questions? Yes?

AUDIENCE: So in the anti-decoder, you have a lot of computations size that were not sequential streams. Like for example, the output of the distinct cosine transform is not a stream of pixel, you are going to have coefficients and things like that. Which are a logical sort of chunk.

BILL THIES: Yes so depending on the granularity of the competition, you don't need to pass individual values over the stream. For example, you can have a stream that inputs the whole array at a time. And so we basically advocate that if you have something that course-grained, you should be passing an array or a macroblock, in the case of MPEG, or a set of coefficients in a structure. So when you have coarse-grain parallelism, you write your program in a course-grained way. The fine-grained things I showed for the bit interleaving and so on, is more for the fine-grained programs. AUDIENCE: Can you do both? In the sense that can you stream over an array, so it's stream of stream, so to speak.

BILL THIES: So there's an interesting multidimensional problem there. Right now we've taken a 1-dimensional approach. So far it's basically the programmer has to set an iteration order, and end up with a 1-dimensional stream coming into and out of every filter. We're working on extending that. Yeah, but when you have basically streams of 2-dimensional data, you like the freedom to either iterate basically in time or in space, depending on what you're doing. And so I think that's more of a research problem. So far we're just been doing a 1-dimensional representation. Yeah good point. Other questions? Yeah?

AUDIENCE: Why did you decide on this synchronous dataflow model as opposed to something more general?

BILL THIES: So our philosophy has been that you want to start with the most kind of basic block of a stream program, and optimize it really well. And then you can stitch those together into higher level blocks. So we think of synchronous dataflow as being kind of the basic block of streaming. You know what's coming in, you know what's coming out. And even if you have a more general model, they'll be pieces that fit under their synchronous dataflow model. And so we saw a lot of optimizations opportunities in there. And really knowing those IO rates can let you do a lot of things that you can't do in a general model. So I wanted to get the simple case right first. And actually kind of our focus now is on expanding, and how do you look at the heterogeneous system, and how do you optimize a more dynamic system. Yep. Other questions? OK. Yes. You can check out our web page. Yeah if you Google for StreamIt, I'm sure you can find it. Yeah, we have a public release. Yes, send us any problems. It's actually it's a good test. We want to make sure it works for everyone. But I mean, we've had, you know, hundreds of downloads. There are a lot of people using StreamIt. It shouldn't break if you download it. Yeah. OK good. Thanks.