Comments (4)
I like stacked bars, so that we can easily compare total times across tools while still getting a feeling of where overhead comes from.
from sok.
We require 10 runs for each benchmark program. Raw values are saved in the CSV file and median/avg is then to be computed during the plotting phase.
@AlexanderViand has existing matplotlib code from a paper for stacked and grouped bars that can be used as basis for our plots.
from sok.
In the S3 directory <timestamp>/plot
there should be files plot_<application>.py
that define a function plot
that takes a list of labels (i..e tool names), a list of pandas dataframes (each tool' *.csv
) and optionally a matplotlib Figure object, and return a Figure containing the desired plot.
As an example, here is plot_cardio.py
:
from typing import List
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
def plot(labels: List[str], pandas_dataframes: List[pd.DataFrame], fig=None) -> plt.Figure:
"""
:param labels:
:param pandas_dataframes:
:param fig:
:return:
"""
# Save current figure to restore later
previous_figure = plt.gcf()
# Set the current figure to fig
if fig is None:
fig = plt.figure()
plt.figure(fig.number)
# Setup Axis, Title, etc
N = len(labels)
plt.title('Runtime for Cardio')
plt.ylabel('Time (ms)')
ind = np.arange(N) # the x locations for the groups
plt.xticks(ind, labels)
width = 0.35 # the width of the bars: can also be len(x) sequence
# Plot Bars
for i in range(N):
df = pandas_dataframes[i]
d1 = df['t_keygen'].mean()
p1 = plt.bar(ind[i], d1, width, color='red')
d2 = df['t_input_encryption'][i].mean()
p2 = plt.bar(ind[i], d2 , width, bottom=d1, color='blue')
d3 = df['t_computation'][i].mean()
p3 = plt.bar(ind[i], d3, width, bottom=d1+d2, color='green')
d4 = df['t_decryption'][i].mean()
p4 = plt.bar(ind[i], d4, width, bottom=d1+d2+d3, color='cyan')
# Add Legend
plt.legend((p4[0], p3[0], p2[0], p1[0]), ('Decryption', 'Computation', 'Encryption', 'Key Generation'))
# Restore current figure
plt.figure(previous_figure.number)
return fig
if __name__ == '__main__':
print("Testing ploting with cardio example")
data = [pd.read_csv('s3://sok-repository-eval-benchmarks/20200729_094952/Cingulata/cingulata_cardio.csv')]
labels = ['Cingulata']
plot(labels, data)
from sok.
There is currently still an issue with the html rendering of the plots on the Visualisation Website.
Apparently, mpld3 does not handle custom labels very well: mpld3/mpld3#360 (comment)
from sok.
Related Issues (20)
- Add Concrete Environment HOT 2
- Design & implement Benchmarks for libraries HOT 5
- Add "naive" versions of manually implemented applications? HOT 1
- Add EVA environment HOT 2
- Implement NN task in EVA
- Add HElib Framework HOT 1
- Check docker_build images workflow task if new images are built (PALISADE, EVA, concrete) HOT 1
- Add docker program example for PALISADE
- Add docker program example for EVA HOT 1
- Implement Chi-Squared Task in EVA HOT 2
- Implement Cardio Task in EVA
- Improve plotting HOT 1
- Move local benchmark script to scripts folder and add minimal documentation HOT 2
- Move dev container script to the scripts folder and add minimal documentation
- Update E3 base image and check whether our programs still work HOT 1
- Add docs on how to inspect a running Docker container HOT 1
- Optimize E3 execution order depending on computation time of the last run HOT 1
- Check why plotting failed in benchmark task HOT 3
- Wiki images broken - HTTP 406 errors HOT 2
- SoK Abbreviation HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from sok.