CSinParallel 1: Using Map-Reduce to Teach Parallel Programming Concepts Across the CS Curriculum
Part 1 -- Fundamentals; Introductory Courses
This first half of the workshop introduces map-reduce computing through the WebMapReduce (WMR)simplified interface to Hadoop, then shares our experience teaching map-reduce and related concepts of parallel and distributed computing to students in introductory sequences.
Introduction to map-reduce computing; demo of WebMapReduce (WMR)
Resources: Why teach map-reduce with WMR?; WMR users guide, including languages included (so far)
Teaching map-reduce with WMR in the introductory sequence: materials, strategies, and experience
Resources: Module; direct link to teaching materials
- Welcome; goals.
Hands-on exercises (7:30pm)
Getting started with WMR
Resources: Intro to WMR module ; see Using WMR , then Counting words with WMR (Python) Data sets on HDFS: /shared/gutenberg/CompleteShakespeare.txt, AnnaKarenina.txt, WarAndPeace.txt; /shared/gutenberg/all/group8
Alternative explorations: WMR code examples in various languages; More WMR code examples
- Getting started with WMR
Part 2 -- Intermediate and Advanced Courses
This part of the workshop uses WMR to explore use of map-reduce computing in more advanced courses, and examines the relationship between the WMR interface and the Hadoop computations it performs.
WebMapReduce and its architecture; obtaining and installing WMR
Resources: WMR sourceforge site; admin page
Examples: using WMR and map-reduce in upper-division (undergraduate) courses and projects. Appropriate uses of map-reduce.
Resources: Module, Concurrency and Map-Reduce Strategies in Various Programming Languages
Overview of the Hadoop implementation of map-reduce.
Resources: Hadoop code for word count ; Hadoop documentation
- WebMapReduce and its architecture; obtaining and installing WMR
Hands-on exercises (9:00)
Assortment of WMR exercises
Resources: Intro to WMR module ; see WMR Activities Movie Data Data sets on HDFS: /shared/MovieLens2, /shared/gutenberg/CompleteShakespeare.txt, AnnaKarenina.txt, WarAndPeace.txt; /shared/gutenberg/all/group8; and code examples in various languages
- Assortment of WMR exercises
Discussion and feedback (9:40)
Please complete our own short survey for grant assessment purposes.
Map-reduce, the cornerstone computational framework for cloud computing applications, has star appeal to draw students to the study of parallelism. Participants will carry out hands-on exercises designed for students at CS1/intermediate/advanced levels that introduce data-intensive scalable computing concepts, using WebMapReduce (WMR), a simplified open-source interface to the widely used Hadoop map-reduce programming environment. WMR supports programming in a choice of languages (including Java, Python, C++, C#, Scheme). Workshop includes brief introduction to direct Hadoop programming. Workshop materials will reside on csinparallel.org, along with WMR software. Intended audience: CS instructors. Laptop required (Windows, Mac, or Linux).