| Home | Overview | License | Download | Publications | Documentation | Help |

PARSEC 3.0 Docs
   
 
Overview

Welcome to PARSEC 3.0 documentation. There are three major changes in PARSEC 3.0:

  • Adding network benchmarks as well as a user-level parallel TCP/IP stack
  • Providing SPLASH-2 and the inputs-enlarged SPLASH-2x.
  • Redesigning the framework for supporting external suites.
This documention presents a quick tutorial on how to run PARSEC 3.0 on real machines as well as simulators.

 

Getting Started

We try to make PARSEC as easy as possible to use. You just need to follow these steps to run PARSEC 3.0:

1. Download PARSEC 3.0


2. Unpack PARSEC 3.0 package

    [user@host ~]$ tar -xzf parsec-3.0.tar.gz
    [user@host ~]$ cd parsec-3.0 
    [user@host ~]$ ls 
    bin CHANGELOG  config CONTRIBUTORS  env.sh  
    ext    FAQ  LICENSE  man  pkgs  README  version
    

3. Setup environment variable before running PARSEC

    [user@host ~]$ source env.sh
    

4. A glimpse of the omnipotent command -- "parsecmgmt"

    [user@host ~]$ parsecmgmt
    [PARSEC] Error: No action specified.
    Usage: parsecmgmt -a ACTION [OPTION]...
      ...
    
    [user@host ~]$ parsecmgmt -a info
    [PARSEC] PARSEC Distribution v3.0
    [PARSEC]
    [PARSEC] PACKAGES
    [PARSEC]
    [PARSEC] parsec.blackscholes [1] (computational finance)
    [PARSEC] Prices a portfolio of options
      ...
    
    [user@host ~]$ parsecmgmt -a status -p all
    [PARSEC] Installation status of selected packages:
    [PARSEC]
    [PARSEC] parsec.blackscholes [1]:
    [PARSEC]   -no installations-
      ...
    

5. Build/run/uninstall benchmarks

    Build and run one benchmark (e.g., streamcluster)
    [user@host ~]$ parsecmgmt -a build -p streamcluster
      ...
    [PARSEC] Copying source code of package parsec.streamcluster.
    [PARSEC] Running 'env version=pthreads /usr/bin/make':
      ...
    
    [user@host ~]$ parsecmgmt -a run -p streamcluster
      ...
    [PARSEC] [---------- Beginning of output ----------]
    PARSEC Benchmark Suite Version 3.0
    read 10 points
    
    real	0m0.016s
    user	0m0.002s
    sys	0m0.001s
    [PARSEC] [----------    End of output    ----------]
      ...
    
    [user@host ~]$ parsecmgmt -a fulluninstall -p streamcluster
    [PARSEC] Removing all installed files:
    [PARSEC] parsec.streamcluster - [Removing installation directory]
    

    Build and run all benchmarks (Note that it may take several hours depending on your machine)
    [user@host ~]$ parsecmgmt -a build -p all
      ...
    
    [user@host ~]$ parsecmgmt -a run -p all
      ...
    [PARSEC] [---------- Beginning of output ----------]
    PARSEC Benchmark Suite Version 3.0
    read 10 points
    
    real	0m0.016s
    user	0m0.002s
    sys	0m0.001s
    [PARSEC] [----------    End of output    ----------]
      ...
    
    [user@host ~]$ parsecmgmt -a fulluninstall -p streamcluster
    [PARSEC] Removing all installed files:
    [PARSEC] parsec.streamcluster - [Removing installation directory]
    

6. Try other inputs (e.g., the inputs for simulation)

    Download the inputs file
    [user@host ~]$ wget http://parsec.cs.princeton.edu/download/3.0/
    parsec-3.0-input-sim.tar.gz
      ...
    
    Move the input file to the same level of PARSEC root directory for unpacking
    [user@host ~]$ mv parsec-3.0-input-sim.tar.gz ~/
    [user@host ~]$ cd ~/
    [user@host ~]$ ls
    parsec-3.0   parsec-3.0-input-sim.tar.gz
    [user@host ~]$ tar -xzf parsec-3.0-input-sim.tar.gz
      ...
    
    Run benchmarks
    [user@host ~]$ parsecmgmt -a run -p streamcluster -i simsmall
      ...
    
    [user@host ~]$ parsecmgmt -a run -p streamcluster -i simlarge -n 2
      ...
    

7. Try more ...

Congratulations! So far, you have been able to successfully use PARSEC 3.0 benchmark suite.

PARSEC 3.0 supports more features. Please refer to the following sections:

Here is a tutorial on PARSEC 3.0:

 

Network Benchmarks

PARSEC 3.0 provides three server/client mode network benchmarks which leverage a user-level TCP/IP stack library for communication. The three network benchmarks are as follows:

Network
Connection
BandwidthPacket
Size
Latency
Sensitive
Region of
Interest
netdedupSingleHighLargeServer
netferretMultipleLowSmallServer
netstreamclusterMultipleHighLargeServer

Note that the three network benchmarks are derived from the existing PARSEC benchmarks:

  • dedup - Next-generation compression with data deduplication
  • ferret - Content similarity search server
  • streamcluster - Online clustering of an input stream

Run network benchmarks on real machines

This figure illustrates the framework for running network benchmarks on real machines. Both server and client use the up-TCP/IP stack library to communicate with each other wherein the underlying communication mechanism is based on shared memory.

Please refer to this link for more details of the library:

Please refer to this link for running network benchmarks on simulators:


Follow these steps to run network benchmarks:

1. Where are the network benchmarks

    [user@host ~]$ pwd
    /home/user/parsec-3.0/pkgs/netapps
    [user@host ~]$ ls
    netdedup  netferret  netstreamcluster
    

2. Build network benchmarks

    build all benchmarks
    [user@host ~]$ parsecmgmt -a build -p netapps
    [PARSEC] Packages to build:  parsec.netdedup parsec.netferret 
    parsec.netstreamcluster parsec.uptcpip
      ...
    

    build one individual benchmark (e.g., netferret)
    [user@host ~]$ parsecmgmt -a build -p netferret
    [PARSEC] Packages to build:  parsec.netferret
      ...
    

3. Run network benchmark (e.g., netferret)

    Run netferret with 'test' input and one network connection (default)
    [user@host ~]$ parsecmgmt -a run -p netferret
    [PARSEC] Packages to build:  parsec.netferret
      ...
    
    Run netferret with 'simlarge' input and one network connection
    (please download the inputs for simulation before this trial)
    [user@host ~]$ parsecmgmt -a run -p netferret -i simlarge
    [PARSEC] Packages to build:  parsec.netferret
      ...
    
    Run netferret with 'simlarge' input, one network connection and multiple threads in server
    [user@host ~]$ parsecmgmt -a run -p netferret -i simlarge -n 2
    [PARSEC] Packages to build:  parsec.netferret
      ...
    
    Run netferret with 'simlarge' input, two network connections and multiple threads in server
    [user@host ~]$ parsecmgmt -a run -p netferret -i simlarge -n 2 -t 2
    [PARSEC] Packages to build:  parsec.netferret
      ...
    

Note that the network benchmarks use a up-TCP/IP stack to communicate which is a library of PARSEC 3.0. The next section will intorduce the up-TCP/IP stack library.

 

up-TCP/IP Stack Library

Motivation

Network applications are ubiquitous, but no existing benchmarks have TCP/IP stack. However, TCP/IP processing is expensive that processing 1Gbps payload needs 1Ghz Pentium CPU power. Thus, we decide to move TCP/IP stack out of kernel in order to enable users to parallelize it for multicores.


Challenges of designing a user-level TCP/IP stack benchmark

  • Similarity - it should behaves similarly as a kernel TCP/IP stack
  • Portability - it can run on both real machines and simulators
  • Flexibility - it can ease the study of TCP/IP stack

TCP/IP stack in PARSEC 3.0

The user-level parallelized TCP/IP (up-TCP/IP) stack of PARSEC 3.0 is ported from the kernel TCP/IP stack of FreeBSD 8.0. The following figure illustrates the diagram of the TCP/IP stack which runs as a library to support communication.


The pink components are almost the same as the FreeBSD's kernel TCP/IP stack except for substituting synchronization primitives (e.g., locks) and removing FreeBSD's user credentials function. The other peripheral dependencies of TCP/IP stack are provided by host service. For example, network interface card (NIC) is emulated by IPC mechanism (shared memory).


Identify TCP/IP regions in network benchmarks

The up-TCP/IP library adopts two APIs, i.e., parsec_enter_tcpip_roi() and parsec_exit_tcpip_roi(), to annotate network benchmarks' source codes so as to indentify TCP/IP regions. After one benchmark's execution completes, the up-TCP/IP library will display an execution time report of the identified TCP/IP regions:

    [user@host ~]$ parsecmgmt -a run -p netstreamcluster -i simlarge
      ...
    =============================================
      Enter PARSEC TCPIP ROI 
    =============================================
    read 16384 points
    [Client:0]: Sending ...
    [Client:0] send bytes = 8388608
    [Client:0] Send data to server ok!
    Receive Ack from Server
    [0] Data size = 8388608B, time =  0.289948s, BW =   28.931MB/s
    Send Ack
    
    
    ------------------------------------------------------------
          Execution Statistics of u-TCP/IP Stack 
    ------------------------------------------------------------
    There are 6 threads, 5 involved to u-TCP/IP stack
    (u-TCP/IP_Threads) / (Total_Threads) = 83.33%
    
    ROI_RUN_TIME =   0.29s
    
         THREAD_TYPE         ACTIVE_TIME           ACTIVE(%)
            Rx_Tcpip              0.016s                5.51
             Tx_Link              0.003s                0.87
             Rx_Link              0.011s                3.74
               Timer              0.000s                0.01
              Socket              0.010s                3.43
    ------------------------------------------------------------
    
    ==============================================
      Exit PARSEC TCPIP ROI 
    ==============================================
      ...
    

Interpret time report

The up-TCP/IP library consists of four internal threads:

  • Rx_Tcpip is responsible for processing incoming TCP/IP packets
  • Tx_Link is responsible for transmiting functionality of NIC driver
  • Rx_Link is responsible for receiving functionality of NIC driver
  • Timer is responsible for handling TCP timers.
User program's threads can be involved in TCP/IP stack because of envoking socket APIs such as uptcp_send() and uptcp_receive(). Thus,
  • Socket indicates those threads involved in up-TCP/IP stack.

Where is the library?

    [user@host ~]$ pwd
    /home/user/parsec-3.0/pkgs/libs/uptcpip
    

Build/uninstall library

    Usually you do not need to explicitly run command to build up-TCP/IP library because those network benchmarks can detect the dependency and automatically call build process for the library. If you indeed want to build the library separately because of modifying the source codes, please run the following command:
    [user@host ~]$ parsecmgmt -a build -p uptcpip
    [PARSEC] Packages to build:  parsec.uptcpip
      ...
    
    However, an explicit command is required to uninstall the library
    [user@host ~]$ parsecmgmt -a fulluninstall -p uptcpip
    [PARSEC] Removing all installed files:
    [PARSEC] parsec.uptcpip - [Removing installation directory]
    

 

SPLASH-2

SPLASH-2 benchmark suite includes applications and kernels mostly in the area of high performance computing (HPC). It has been widely used to evaluate multiprocessors and their designs for the past 15 years.

Recent studies show that SPLASH-2 and PARSEC benchmark suites complement each other well in term of diversity of architectural characteristics such as instruction distribution, cache miss rate and working set size.

In order to provide computer architects with the convenient use of both benchmarks, we have integrated SPLASH-2 into the PARSEC environment in this release. Users can now build, run and manage both workloads under the same environment framework.


1. Check the status of SPLASH-2

    [user@host ~]$ parsecmgmt -a status -p splash2
      ...
    

2. Build SPLASH-2 benchmarks

    Build all SPLASH-2 benchmarks

    [user@host ~]$ parsecmgmt -a build -p splash2
      ...
    
    Build one SPLASH-2 benchmark (e.g., fft)

    [user@host ~]$ parsecmgmt -a build -p splash2.fft
      ...
    

2. Run SPLASH-2 benchmarks

    Run all SPLASH-2 benchmarks

    [user@host ~]$ parsecmgmt -a run -p splash2
      ...
    
    Run one SPLASH-2 benchmark (e.g., fft)

    [user@host ~]$ parsecmgmt -a run -p splash2.fft
      ...
    
    Note that SPLASH-2 benchmark supports one input, i.e., "test" inputs

    [user@host ~]$ parsecmgmt -a run -p splash2.fft -i simlarge
      ...
    [PARSEC] NOTE: SPLASH-2 only supports "test" input sets.
    [PARSEC]       Please use SPLASH-2x for multiple input sets.
      ...
    

3. Uninstall SPLASH-2 benchmarks

    Uninstall all SPLASH-2 benchmarks

    [user@host ~]$ parsecmgmt -a fulluninstall -p splash2
      ...
    
    Uninstall one SPLASH-2 benchmark (e.g., fft)

    [user@host ~]$ parsecmgmt -a fulluninstall -p splash2.fft
      ...
    

 

SPLASH-2x

SPLASH-2x has several input datasets at different scale. To scale up the input sets for SPLASH-2, we have explored the input space of the SPLASH-2 workloads to generate multiple scales of input sets, similar to PARSEC's criterion:

  • Native (< 15 minutes)
  • Simlarge (<16 seconds)
  • Simmedium (<4 seconds)
  • Simsmall (<1 second)

Please refer to this memo for more details on SPLASH-2x.


Run SPLASH-2x benchmarks

    Use the similar commonds as running SPLASH-2 benchmarks by just replacing "splash2" with "splash2x"

    [user@host ~]$ parsecmgmt -a run -p splash2x
      ...
    
    Run one SPLASH-2 benchmark (e.g., fft)

    [user@host ~]$ parsecmgmt -a run -p splash2x.fft
      ...
    
    Note that SPLASH-2x benchmark supports multiple inputs

    [user@host ~]$ parsecmgmt -a run -p splash2x.fft -i simlarge
      ...
    

 

Run PARSEC on Simulators

There are two ways to run PARSEC 3.0 on simulators:

  • Full-System Simulators: such as Simics, GEM5.
  • Trace-Driven Simulators: such as some Pin-based simulators.
Except for network benchmarks, you can run normal parsecmgmt command within simulators. However, it requires some effort to run network benchmarks on simulators.

Run network benchmarks on full-system simulators

This figure illustrates the framework for running network benchmarks on a full-system simulator. In this mode, server runs on simulator and client runs on a real machine. Server and client communicate via a simulated network link provided by the simulator.

Next, let's demonstrate a concrete case.

1. Preparation

You need to make a full-system simulator ready. The following configuration is used in our case.

Description
SimulatorSimics-4.2
Simulated ArchitectureX86_64
Simulated OSCentOS-5.4 (Linux 2.6.18)
Disk image nameparsec-machine-0.image
Host ArchitectureX86_64
Host OSX86_64
Host CompilerGCC 4.2.4

2. Make PARSEC ISO image

    [user@host ~]$ cd ~/parsec-3.0
    [user@host ~]$ source env.sh
    [user@host ~]$ parsecmgmt -a build -p netapps
      ...
    
    [user@host ~]$ cd ..
    [user@host ~]$ mkisofs -l -D parsec-3.0 > parsec3.iso
      ...
    [user@host ~]$ mv parsec3.iso your-simics-dir/your-workspace/
    targets/x86-440bx/
    

3. Write Simics script

An sample sciprt can be downloaded from here:

    [user@host ~]$ cd your-simics-dir/your-workspace/targets/x86-440bx/ 
    [user@host ~]$ wget http://parsec.cs.princeton.edu/download/other/
    dredd-parsec.simics
      ...
    

4. Run network benchmark's server on simulator

    [user@host ~]$ cd your-simics-dir/your-workspace/targets/x86-440bx/ 
    [user@host ~]$ ../../simics -stall dredd-parsec.simics
      ...
    

4. Run client on real machine Open a new terminator and run

    [user@host ~]$ cd parsec-3.0
    [user@host ~]$ source env.sh
    [user@host ~]$ parsecmgmt -a run -p netdedup -m client
      ...
    




Copyright 2007-2009 Princeton University. All Rights Reserved.