Interactive Development with Jupyter

For exploratory data analysis, rapid prototyping, or showcasing examples, you can write and execute Renoir data streams interactively in a Jupyter Notebook using Rust's evaluation kernel.

Prerequisites

Before setting up Jupyter, ensure you have the Rust toolchain installed. If not, follow the Installation Guide.

1. Install Evcxr Jupyter Kernel

Evcxr is a Rust evaluation context providing a REPL (similar to Python's IPython) and a Jupyter Kernel.

To install the kernel, follow these steps:

Install the evcxr_jupyter binary:
```
cargo install --locked evcxr_jupyter
```
Register the Jupyter Kernel:
```
evcxr_jupyter --install
```
:octicons-light-bulb-16: Ensure that Cargo's binary folder ($HOME/.cargo/bin) is in your environment PATH variable so Jupyter can locate the evcxr_jupyter executable.

2. Using the Jupyter Kernel in VS Code

You can run Jupyter Notebooks directly inside Visual Studio Code:

Install Jupyter in your Python environment: Make sure you have python and the standard jupyter package installed:
```
pip install jupyter
```
Install the VS Code Extensions: Install the VS Code Jupyter Extension.
Create a Notebook: Create a new file with the extension .ipynb (e.g., renoir_demo.ipynb) and open it in VS Code.
Select the Rust Kernel:
- Click Select Kernel in the top-right corner of the editor.
- Choose Jupyter Kernel...
- Select the Rust kernel installed by evcxr_jupyter in the previous step.

3. Importing Renoir & Jupyter Directives

In an Evcxr-powered cell, you can import dependencies and configure compilation settings using special colon-prefixed directives.

Key Evcxr Directives

:dep: Imports external crates from crates.io or Git repositories. Follows the same syntax as a dependency line in Cargo.toml.
```
:dep renoir = { git = "https://github.com/deib-polimi/renoir" }
```
:cache: Sets a compilation cache size (in MiB). Caching compilation segments speeds up interactive executions.
```
:cache 500
```
:opt: Sets the optimization level (0 to 3). While level 0 compiles fastest, level 2 or 3 runs the streaming dataflow much faster.
```
:opt 2
```
:vars: Prints a table listing all variables in scope alongside their types and values.
```
:vars
```

4. Complete Interactive Example

Add the following snippets to consecutive cells in your Jupyter Notebook.

Cell 1: Setup and Imports

:cache 500
:opt 2
:dep renoir = { git = "https://github.com/deib-polimi/renoir" }

use renoir::prelude::*;

Cell 2: Creating a Local Dataflow

let ctx: StreamContext = StreamContext::new_local();

// Compute squares for a range of integers in parallel
let result: StreamOutput<Vec<(i32, i32)>> = ctx.stream_par_iter(0..100)
    .map(|x| (x, x * x))
    .collect_vec();

// Execute the pipeline synchronously 
ctx.execute_blocking();

// Retrieve the collected results
let output: Option<Vec<(i32, i32)>>  = result.get();

Cell 3: Print and Inspect

// Display variables currently defined in the session
:vars

Variable	Type	Value
`output`	`Option<Vec<(i32, i32)>>`	`Some(...)`

Cell 4: Display Results

println!("First 5 squared pairs: {:?}", &output.as_ref().unwrap()[0..5]);

Expected Cell Output:

First 5 squared pairs: [(0, 0), (1, 1), (2, 4), (3, 9), (4, 16)]