lschoe / mpyc Goto Github PK
View Code? Open in Web Editor NEWMPyC: Multiparty Computation in Python
License: MIT License
MPyC: Multiparty Computation in Python
License: MIT License
Running a simple program:
#mpyc_tests.py
from mpyc.runtime import mpc
from mpyc.random import randint
print(f"M: {len(mpc.parties)} t:{mpc.threshold} pid:{mpc.pid}")
secint = mpc.SecInt()
mpc.run(mpc.start())
b_random = randint(secint,0,10)
print(f"Random int is : {mpc.run(mpc.output(b_random))}")
Run above without '--index' or '-I' flags works as expected:
$ python mpyc_tests.py -M8 -t2
M: 8 t:3 pid:0
2019-09-25 21:48:36,739 Start MPyC runtime v0.5.10
2019-09-25 21:48:36,964 All 8 parties connected.
Random int is : 5
Run with flags, sets the pid as expected but causes program to hang
$ python mpyc_tests.py -M8 -t2 -I1
M: 8 t:3 pid:1
2019-09-25 21:50:09,798 Start MPyC runtime v0.5.10
Excuse me, but i don't understand how this library works.
There is "Share" class. It is " A secret-shared value. ..." as you say, but i see only some wrapper over numbers i don't understand. I don't see process of number sharing or circuits building or something else to MPC related
You wrap integeres into secint, but how could it help to remain value of sorting integers in secret?
If some entity will have gotten x array to perform oddeven_merge_sort() func on its comp it anyway will have can reconstruct true values of sorting numbers. So, i don't understand how that solution provide confidentiality to sorting numbers :c
And i dont understand multiparty case of work. As i see if i setup it there is no changing in algorithm work. But MPC implies a lot of different communications
Please, explain me what part of MPC issue your library solves. c:
Hello, sorry for disturbing you.
I tried to use comparison on SecFld type and found that only operator supported is '=='. For the other operators, it will return a type error.
TypeError: '>=' not supported between instances of 'SecFld34(GF(8589934609))' and 'SecFld34(GF(8589934609))'
Sorry for my lack of knowledge on this. I am curious why any other operators like '>=' are not supported. In addition, the documentation mentions that Secure number types ensures that operators such as +,*,>= are defined by operator overloading. So, does it mean SecFld is not a Secure number type?
Hi, I'm leaning mpyc, in demo cnnmnist.py, the para f=6, and force python integer to secnum,
scale = lambda a: secnum(int(round(a * f))) # force Python integers;
why f is set to 6? I test f = 1, 2, the demo also work fine.
on this line:
from kmsurvival import fit_plot, events_to_table, events_from_table, logrank_test, aggregate, agg_logrank_test
stark sm_spencer$ cat km_analysis.py
import os, functools
import pandas as pd
import matplotlib.pyplot as plt
import lifelines.statistics
from mpyc.runtime import mpc
mpc.logging(False)
from kmsurvival import fit_plot, events_to_table, events_from_table, logrank_test, aggregate, agg_logrank_test
print('Hello MPC World')
stark sm_spencer$ python3 km_analysis.py
Traceback (most recent call last):
File "km_analysis.py", line 7, in <module>
from kmsurvival import fit_plot, events_to_table, events_from_table, logrank_test, aggregate, agg_logrank_test
ImportError: cannot import name 'fit_plot' from 'kmsurvival' (/usr/local/lib/python3.7/site-packages/kmsurvival/__init__.py)
stark sm_spencer$
I took a quick look at the kmsurvival source and didn't see anything named fit_plot... am I missing something?
I realised running localhost computations using -M
>= 20 number of parties is extremely slow (From 2 to 15, I measured the time it took, and it reasonably quadratically grows, but after 20, it explodes. Do you know whether this is a limitation of the library, or rather a problem on my side (hardware specs, etc...?)
Hi, I have seen examples of inference in the code base, however what changes will I need to do to use it for training (with back propagation etc) , is there any example for it ?
Hi,
I'm trying to use the library for some simple aggregations over pandas dataframes, where each row is a computing party, to conduct some runtime experiments for MPC. As you explained in detail in #4, this will have some limitations due to PRSS.
I noticed, that it is possible to run it without PRSS (--no-prss flag). As far as I understand from your code, this is a simple Shamir sharing with random coefficient polynomials as implemented in thresha.random_split()
.
However, I can't see from the code where PRSS would actually have an influence.
While the computation time decrease as expected, the communication costs in bytes are the same with and without PRSS as you can see from the outputs below. Only the time used for setting up the connects increases. Does this imply the PRSS shares are created during connection? I thought PRSS actually is used to decrease communication costs.
I assume 22 bytes is for sending the share to all other parties and then another 22 bytes to reconstruct the secret with T parties. Is that correct?
I would appreciate if you could explain me the effects of using the --no-prss flag.
With PRSS:
python addition.py -M50 -T4 --log-level=debug
2023-06-09 16:05:35,123 Start MPyC runtime v0.9
2023-06-09 16:06:01,465 All 50 parties connected.
Total : 35890
2023-06-09 16:06:16,771 Stop MPyC -- elapsed time: 0:00:15.262|bytes sent: 1166
2023-06-09 16:06:16,772 Bytes sent per party: 0 44 44 44 44 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22
2023-06-09 16:06:16,773 Synchronize with all parties before shutdown
2023-06-09 16:06:16,776 Closing connections with other parties
Execution time including connecting: 0:00:41.692
Without PRSS:
python addition.py -M50 -T4 --no-prss --log-level=debug
2023-06-09 16:05:17,697 Start MPyC runtime v0.9
2023-06-09 16:05:18,880 All 50 parties connected.
Total: 35890
2023-06-09 16:05:18,935 Stop MPyC -- elapsed time: 0:00:00.054|bytes sent: 1166
2023-06-09 16:05:18,935 Bytes sent per party: 0 44 44 44 44 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22
2023-06-09 16:05:18,936 Synchronize with all parties before shutdown
2023-06-09 16:05:18,941 Closing connections with other parties
Execution time including connecting: 0:00:01.263
How to use the class SecureArray? It is not seem in the document? I want to transfer a np.ndarray type to secure type, how can I do that?
I write a program that needs to access an array by secret index. Does MPyC contain ORAM implementation?
Hello,
I've run some experiments where I sort "medium-size" secure lists (i.e., around 1K elements). For example, see the following snippet:
async def main():
sectype = mpc.SecInt(64)
await mpc.start()
if mpc.pid == 0:
rand_list = [random.randint(0, 1024) for _ in range(1000)]
else:
rand_list = None
rand_list = await mpc.transfer(rand_list, senders=0)
sec_list = [sectype(i) for i in rand_list]
sorted = mpc.sorted(sec_list)
print(await mpc.output(sorted))
await mpc.shutdown()
If I execute my script with no argument or with -M0
, the script finishes without any issue. If I use -M3
or even -M1
, the Python process takes more and more memory. Sometimes, it even fills my memory entirely for larger lists or larger M. On the other hand, with -M0
, the memory footprint is constant (and much smaller).
I suppose this behaviour comes from the secret sharing costs but I wonder whether I can optimize it to reduce the memory footprint. For example, I see several nested loops in the _sort()
function and wonder whether the coroutine stores the "comparison shares" from all comparison rounds until the end instead of releasing some of them round by round.
To sum up, is there a simple trick I missed in the documentation or the code to optimize the memory footprint in these case?
Hi, below part is from shuffle function. Are all the lines necessary or is it just to increase the randomness? x_u is already random. What is the purpose of scalar_mul and vector_add? Can't we get rid of them to increase the performance of shuffling by decreasing the execution time?
Also could you name the paper, study that shuffling algorithm originated?
for i in range(n-1):
u = random_unit_vector(sectype, n - i)
x_u = runtime.in_prod(x[i:], u)
d = runtime.scalar_mul(x[i] - x_u, u)
x[i] = x_u
x[i:] = runtime.vector_add(x[i:], d)
return
Excuse me,
I noticed that performing division by a negative secure integer in the current implementation results in a random number instead of a specified outcome. I'm curious about the reasons behind this lack of support for division by negative secure integers. It seems like a straightforward operation that could be achieved by dividing a positive value and negating the result. Is the determination of the sign of the variable computationally expensive, leading to the exclusion of a dedicated implementation for division by negative secure integers?
Additionally, I'm interested in understanding why the result obtained is a random number rather than a fixed incorrect value.
Hi,
While experimenting with some arrays of fixed-point numbers, I discovered two problems with the division operator:
Calling np.divide
on SecureFixedPointArray will call _norm
. This function will fail on the following instruction:
l = type(a).bit_length
I suppose this function works on scalars but fails on secure arrays since they have no bit_length
attribute. I suppose we can quickly fix this by taking the bit_length of any element in the secure array, but I let you tell me whether you would prefer a more general refactoring to solve this issue.
To circumvent this problem, I tried to call directly np.reciprocal on my array. However, it seems like there is a numerical issue in this function. For example, with the following inputs [ 1. 36. 1. 16. 49. 1. 81. 49. 64. 121.]
, I obtain [3.73109370e+28 -3.68142988e+28 3.73109370e+28 2.55816413e+28 -3.71193020e+28 3.73109370e+28 3.82035502e+28 -3.71193020e+28 -1.00870767e+28 -1.77230424e+28]
.
I haven't investigated the origin of this behaviour yet.
As for my previous PRs, if we agree on a fix, I can implement them and submit a PR.
Suppose I run the following program on three parties, with input hardcoded purely to demonstrate this issue:
from mpyc.runtime import mpc
async def main():
await mpc.start()
a0 = mpc.input(mpc.SecInt(8)(300))
a1 = a0[0] + a0[1] + a0[2]
print(await mpc.output(a1))
await mpc.shutdown()
mpc.run(main())
(I'm executing this program on localhost in three separate shells via python prog.py -M 3 -I {0,1,2} -B 8000
, Python 3.10.5.)
At first glance, I would expect this program to either:
SecInt(8)
with a value that overflows the type's nominal range,output
the result of the overflowing addition, or-124
(i.e. (300 + 300 + 300) & 0xff
interpreted as a two's-complement signed value)Instead, the program produces the result 900
. Should I expect this behavior, or is there some way I can leverage the library's other types to achieve one of the potential behaviors I listed above?
Hello, when I use mpc.or_ for bitwise OR. I find the result unexpected. It seems that mpc.or_ works only for 1-bit values. However, for mpc.and_, I find the result works successfully. From #36, it seems that you have provided an implementation for bitwise OR with the full bit length. I want to ask whether it is feasible to change the implementation of or_ in runtime.py to the same with #36.
def or_(self, a, b):
x = self.to_bits(a)
y = self.to_bits(b)
return self.from_bits(self.vector_sub(self.vector_add(x, y), self.and_(x, y))))
With this change, it will help me use bitwise OR and bitwise AND more conveniently.
In addition, it seems that the '|' has been overloaded for secure integers with 1-bit, so the function of mpc.or_ and '|' looks repetitive. I think mpc.or_ should release this limitation just like mpc.and_.
Hi,
I'm trying to write a simple program that shares a number 50
with two parties and then accesses those shares and prints them out.
This is my code:
from mpyc.runtime import mpc
async def main():
async with mpc:
secnum = mpc.SecInt()
a = secnum(50 if mpc.pid == 0 else None)
s = mpc.input(a, 0)
share = await mpc.gather(s)
print(await mpc.output(s))
print(share)
print(f'Share of s of party {mpc.pid}: {s.share}')
if __name__ == "__main__":
mpc.run(main())
mpc.run(mpc.shutdown())
My output is as follows:
Party 0
50
50
Share of s of party 0: 50
Party 1
50
50
Share of s of party 1: 50
I would expect share
or s.share
to be the actual share that that party has, not the original value. How can I access the share for every party in plaintext? I've tried looking in the demos, but I'm still confused. Thank you!
I tried to improve the performance using the code in np_*.py
.
I have encountered a format issue TypeError: can't multiply sequence by non-int of type 'ArraySecInt42'
.
T = secint.array(np.ones(n, dtype='O'))
S = [secint.array(np.array([[t[i] == j for t in transactions] for j in attr_ranges[i]]))
for i in range(d)]
S_A = [[mpc.in_prod([S[k][j][l] for k in RR], ii) for l in range(len(S[0][0]))] for j in range(ell)]
T_SA = T * S_A # mpc.schur_prod
S_A
is a list of normal SecInt data, while T
is np format. S_A
is 2D list.
I tried to use np.concatenate
etc. to convert S_A
but failed.
The normal version without using np has been tested successfully.
If you want to test the np version to check the error, I can provide the code
Hi,
The core code for unaimous.py is as follows:
mpc.run(mpc.start())
votes = mpc.input(secbit(vote), senders=voters)
result = mpc.run(mpc.output(mpc.all(votes), receivers=voters))
mpc.run(mpc.shutdown())
I tried a two-party instance. For one party, after executing
result = mpc.run(mpc.output(mpc.all(votes), receivers=voters))
,
I find that this party could see all votes from the variable vates
, so how to understand the privacy of this demo?
Hey lschoe, I'm noticing some incorrect behavior for bitwise XOR on SecInt
s.
If I run the following program (via python p0.py -M 2
, for example)...
from mpyc.runtime import mpc
async def main():
await mpc.start()
a0 = mpc.input(mpc.SecInt(8)(3))
a1 = mpc.xor(a0[0], a0[1])
print(await mpc.output(a1))
await mpc.shutdown()
mpc.run(main())
...I see 6
as the output, rather than the expected 0
.
Digging around a little, I see that xor
is implemented as (apparently arithmetic) addition, which would account for this behavior.
I also see that __xor__
on SecureNumber
s is "for now 1-bit only.". Do you have plans to relax this restriction?
Is there another operation I should be using to accomplish bitwise XOR on values of multiple bits?
Thanks!
Hi Berry,
Sorry for disturbing you again, but I was wondering how MPyC sets field.modulus
and field.order
used to create the Shamir Polynomials in thresha.random_split()
I saw that they are set to be the same in finfields.pGF
and that the prime find_prime_root(l)
with bit length l=bit_length+sec_param, defaulting to get_arg_parser()
.
But not sure if that is really the case?
Thanks for a clarification.
Hello,
is it possible to have an architecture as follows: There are "Input Parties"
Each of the Input Parties have one secret input. They split the input into
I skimmed the demos and found https://github.com/lschoe/mpyc/blob/master/demos/unanimous.py, in which the sets of Input Parties and Computation Parties differ. However, I'm unsure whether or not an architecture as I described would be possible with MPyC.
Is this possible, and if yes, what methods can be used for this?
I'd appreciate your advice! Thank you very much for providing library as Open Source, helps me a lot!
Hi,
I am trying to implement a simple function which outputs the correlation coefficient on secret shared values. This function uses multiple mpc.add, mpc.sub, mpc.mul and mpc.div. I have verified that my implementation is correct.
The issue is that since every one of these mpc maths functions causes a slight loss in precision, all of these losses add up and the function output is quite different from the correlation coefficient obtained from the correlation coefficient obtained from the plaintext. For example:
Plainly computed pearson
-0.6625992397006207
Computed in 0.0014712810516357422 seconds
Securely computed pearson
-0.5963393168058246
Computed in 0.24330759048461914 seconds
Error between values -0.0662599228947961
Do you know if there is any way to mitigate this, please? I am relatively new to MPyC so I may be missing something obvious!
Thanks!
All the best,
Joao
It appears that the conversion from (signed) SecFld objects to other objects yields incorrect results when the shared secret has a negative value.
from mpyc.runtime import mpc
async def main():
await mpc.start()
secint = mpc.SecInt()
secfld = mpc.SecFld(min_order=2**10, signed=True)
a = secint(3)
a = mpc.input(a, 0)
print(await mpc.output(a))
b = mpc.convert(a, secfld)
print(await mpc.output(b))
c = mpc.convert(b, secint)
print(await mpc.output(c))
await mpc.shutdown()
if __name__ == "__main__":
mpc.run(main())
Result:
2020-03-03 16:55:50,690 Install package gmpy2 for better performance.
2020-03-03 16:55:50,691 Start MPyC runtime v0.6
-3
-3
1028
2020-03-03 16:55:50,700 Stop MPyC runtime -- elapsed time: 0:00:00.009001
Of course, we'd expect the result to be -3. Help would be appreciated!
Hi lschoe, thanks for such a great library! I have a question about the most efficient way to perform an XOR using SecureArrays
.
My problem is as follows: I would like to securely generate n
random numbers in [0,1) by having two participants generate n x k
bits, perform a secure XOR on each k
-bit array and then convert this into n
secure floats to use in other computations.
For simplicity let's just assume I want to generate uniform random integers. From my understanding in #36 the only supported way to XOR SecureIntegers
is to work directly with bits using mpc.to_bits()
and mpc.from_bits()
or in my case the np
versions.
My code is the following:
from mpyc.runtime import mpc
import numpy as np
async def main():
await mpc.start()
n = 2 # Two random ints
secint = mpc.SecInt()
random_ints = np.random.random_integers(0, 2**31, size=n)
shares = mpc.input(mpc.np_to_bits(secint.array(random_ints))) # Shape (n, k=32)
xor_arrs = shares[0] + shares[1] - 2*(shares[0] * shares[1]) # xor
r = mpc.np_from_bits(xor_arrs) # Shape (n,) SecInt32
print(await mpc.output(r))
await mpc.shutdown()
mpc.run(main())
This currently errors with:
Traceback (most recent call last):
File "/Users/samuelmaddock/Documents/GitHub/examples/issue.py", line 17, in <module>
mpc.run(main())
File "/Users/samuelmaddock/Documents/GitHub/mpyc/mpyc/runtime.py", line 184, in run
return self._loop.run_until_complete(f)
File "/opt/homebrew/Caskroom/miniforge/base/envs/synth/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
return future.result()
File "/Users/samuelmaddock/Documents/GitHub/examples/issue.py", line 10, in main
shares = mpc.input(mpc.np_to_bits(secint.array(random_ints))) # Shape (n, k=32)
File "/Users/samuelmaddock/Documents/GitHub/mpyc/mpyc/runtime.py", line 386, in input
y = self._distribute(x, senders)
File "/Users/samuelmaddock/Documents/GitHub/mpyc/mpyc/asyncoro.py", line 414, in typed_asyncoro
coro.send(None)
File "/Users/samuelmaddock/Documents/GitHub/mpyc/mpyc/runtime.py", line 428, in _distribute
x = x[0].value.flat # indexable iterator
AttributeError: 'NoneType' object has no attribute 'value'
This is because mpc.np_to_bits()
is only supported for a SecureFiniteField
and returns None
when I pass it an ArraySecInt32
.
Is there a way I can rewrite this code (or modify np_to_bits
) to apply to integers? I tried working directly with SecureFiniteField
but I couldn't find a way to securely cast between array types?
Hello, sorry for disturbing you.
I want to conduct a bitwise on two secure integers. I observed that the mpc._or is limited to 1-bit only, so I try to use 1-bit secure integer but the result was still unexpected. I also tried the methods mentioned in #36, it works but may be a bit complex.
The coding is shown below.
from mpyc.runtime import mpc
async def main():
secint = mpc.SecInt(1)
a = secint(1)
b = secint(1)
x = mpc.to_bits(a)
y = mpc.to_bits(b)
z1 = mpc.schur_prod(x, y) # bitwise and
z2 = mpc.vector_sub(mpc.vector_add(x, y), z1)
c1 = mpc.from_bits(z1)
c2 = mpc.from_bits(z2)
print("c2", await mpc.output(c2))
bitor = mpc.or_(a, b)
print("bitor", await mpc.output(bitor))
mpc.run(main())
The result is shown below.
2023-05-30 16:47:52,031 Install package numpy for more functionality.
2023-05-30 16:47:52,031 Install package gmpy2 for better performance.
c2 1
bitor 3
a
and b
are two 1-bit secure integers, but the result is still unexpected. Does it mean I have to implement the or like #36?
I am sorry to bother you. I have some content that I don't understand, when I read your project source code. I hope I can get your help. I don't understand how this part works๏ผ
if not EQ: # a la Toft
s_sign = (await self.random_bits(Zp, 1, signed=True))[0].value
e = [None] * (l+1)
sumXors = 0
for i in range(l-1, -1, -1):
c_i = (c >> i) & 1
r_i = r_bits[i].value
e[i] = Zp(s_sign + r_i - c_i + 3*sumXors)
sumXors += 1 - r_i if c_i else r_i
e[l] = Zp(s_sign - 1 + 3*sumXors)
g = await self.is_zero_public(stype(self.prod(e)))
h = 3 - s_sign if g else 3 + s_sign
z = (c - a_rmodl + (h << l-1)) / (1<<l)
Questions:
Looking forward to your reply.
Hi,
Sorry to bother you but I bumped into a behaviour I cannot explain. To put it simply, I am doing some secure linear algebra experiments. I start from a random numpy matrix and transform it into a list of secure numbers to compute some stuff.
Initially, I tested everything in the 1-party mode. All my results were correct.
Today, I changed the number of parties to M=3. All my results are now false. When M=1, my secure list have coherent values but when M=3, the values are all incorrect. I suppose that I have done a mistake in the transformation from numpy object to secure list but I am not sure were. I struggle to pin point where is my mistake. The data transformation looks like this:
for i in range(mat.shape[0]): # for a vector
sec_list.append(sectype(mat[i,0]))
# It is not really a sample of my codebase but it summarizes the logic
If ever you have a little nudge for me, I would be glad to hear it!
PS: in addition to the correct result, one of my coroutine even crashes with the following traceback. Something seems definitely wrong with my share generation...
Traceback (most recent call last):
File "/usr/lib/python3.10/asyncio/events.py", line 80, in _run
self._context.run(self._callback, *self._args)
File "/home/mdamie/.local/lib/python3.10/site-packages/mpyc/asyncoro.py", line 429, in <lambda>
task.add_done_callback(lambda t: _reconcile(decl, t))
File "/home/mdamie/.local/lib/python3.10/site-packages/mpyc/asyncoro.py", line 353, in _reconcile
givn = task.result()
File "/home/mdamie/.local/lib/python3.10/site-packages/mpyc/asyncoro.py", line 281, in _wrap_in_coro
return await awaitable
File "/home/mdamie/.local/lib/python3.10/site-packages/mpyc/asyncoro.py", line 269, in __await__
val = self.coro.send(None)
File "/home/mdamie/.local/lib/python3.10/site-packages/mpyc/runtime.py", line 572, in _reshare
in_shares = thresha.random_split(field, x, t, m)
File "/home/mdamie/.local/lib/python3.10/site-packages/mpyc/thresha.py", line 32, in random_split
T_is_field = isinstance(s[0], field) # all elts assumed of same type
IndexError: index 0 is out of bounds for axis 0 with size 0
Traceback (enclosing MPyC coroutine call):
File "/home/mdamie/.local/lib/python3.10/site-packages/mpyc/runtime.py", line 956, in np_multiply
return c
Traceback (most recent call last):
File "/home/mdamie/research/secure-sparse-computations/experiments.py", line 557, in <module>
mpc.run(main())
File "/home/mdamie/.local/lib/python3.10/site-packages/mpyc/runtime.py", line 184, in run
return self._loop.run_until_complete(f)
File "/usr/lib/python3.10/asyncio/base_events.py", line 644, in run_until_complete
raise RuntimeError('Event loop stopped before Future completed.')
RuntimeError: Event loop stopped before Future completed.
Hi,
I am experimenting some large matrix multiplications and I ran into an unexpected issue. To put it simply, I am multiplying two large matrices with 100 rows and an increasing number of columns. Until 3K columns, everything is fine but for 3K columns and more, I have an error I don't understand:
Traceback (most recent call last):
File "/usr/lib/python3.9/asyncio/events.py", line 80, in _run
self._context.run(self._callback, *self._args)
File "/home/mdamie/sparsesecureml/venv/lib/python3.9/site-packages/mpyc/asyncoro.py", line 431, in <lambda>
task.add_done_callback(lambda t: _reconcile(decl, t))
File "/home/mdamie/sparsesecureml/venv/lib/python3.9/site-packages/mpyc/asyncoro.py", line 355, in _reconcile
givn = task.result()
File "/home/mdamie/sparsesecureml/venv/lib/python3.9/site-packages/mpyc/asyncoro.py", line 283, in _wrap_in_coro
return await awaitable
File "/home/mdamie/sparsesecureml/venv/lib/python3.9/site-packages/mpyc/asyncoro.py", line 271, in __await__
val = self.coro.send(None)
File "/home/mdamie/sparsesecureml/venv/lib/python3.9/site-packages/mpyc/runtime.py", line 3672, in np_random_bits
_r = thresha.np_pseudorandom_share(field, m, self.pid, prfs, self._prss_uci(), h)
File "/home/mdamie/sparsesecureml/venv/lib/python3.9/site-packages/mpyc/thresha.py", line 171, in np_pseudorandom_share
s = sum(prf_S(uci, (n,)) * _f_S_i(field, m, i, S)
File "/home/mdamie/sparsesecureml/venv/lib/python3.9/site-packages/mpyc/thresha.py", line 171, in <genexpr>
s = sum(prf_S(uci, (n,)) * _f_S_i(field, m, i, S)
File "/home/mdamie/sparsesecureml/venv/lib/python3.9/site-packages/mpyc/thresha.py", line 257, in __call__
dk = shake_128(self.key + s).digest(n_ * l)
ValueError: [digital envelope routines: EVP_DigestFinalXOF] not XOF or invalid length
File "/home/mdamie/sparsesecureml/venv/lib/python3.9/site-packages/mpyc/runtime.py", line 795, in np_trunc
r_bits = await self.np_random_bits(Zp, f * n)
Traceback (enclosing MPyC coroutine call):
2023-10-02 08:01:11,376 Exception in callback _SelectorSocketTransport._call_connection_lost(ConnectionRes...eset by peer'))
handle: <Handle _SelectorSocketTransport._call_connection_lost(ConnectionRes...eset by peer'))>
Traceback (most recent call last):
File "/usr/lib/python3.9/asyncio/events.py", line 80, in _run
self._context.run(self._callback, *self._args)
File "/usr/lib/python3.9/asyncio/selector_events.py", line 978, in _call_connection_lost
super()._call_connection_lost(exc)
File "/usr/lib/python3.9/asyncio/selector_events.py", line 736, in _call_connection_lost
self._protocol.connection_lost(exc)
File "/home/mdamie/sparsesecureml/venv/lib/python3.9/site-packages/mpyc/asyncoro.py", line 147, in connection_lost
raise exc
File "/usr/lib/python3.9/asyncio/selector_events.py", line 856, in _read_ready__data_received
data = self._sock.recv(self.max_size)
ConnectionResetError: [Errno 104] Connection reset by peer
2023-10-02 08:01:11,376 Exception in callback _SelectorSocketTransport._call_connection_lost(ConnectionRes...eset by peer'))
handle: <Handle _SelectorSocketTransport._call_connection_lost(ConnectionRes...eset by peer'))>
Traceback (most recent call last):
File "/usr/lib/python3.9/asyncio/events.py", line 80, in _run
self._context.run(self._callback, *self._args)
File "/usr/lib/python3.9/asyncio/selector_events.py", line 978, in _call_connection_lost
super()._call_connection_lost(exc)
File "/usr/lib/python3.9/asyncio/selector_events.py", line 736, in _call_connection_lost
self._protocol.connection_lost(exc)
File "/home/mdamie/sparsesecureml/venv/lib/python3.9/site-packages/mpyc/asyncoro.py", line 147, in connection_lost
raise exc
File "/usr/lib/python3.9/asyncio/selector_events.py", line 856, in _read_ready__data_received
data = self._sock.recv(self.max_size)
ConnectionResetError: [Errno 104] Connection reset by peer
Traceback (most recent call last):
File "/home/mdamie/sparsesecureml/venv/bin/benchmark", line 11, in <module>
load_entry_point('securesparsecomputations', 'console_scripts', 'benchmark')()
File "/home/mdamie/sparsesecureml/securesparsecomputations/benchmark.py", line 410, in run
mpc.run(main())
File "/home/mdamie/sparsesecureml/venv/lib/python3.9/site-packages/mpyc/runtime.py", line 177, in run
return self._loop.run_until_complete(f)
File "/usr/lib/python3.9/asyncio/base_events.py", line 640, in run_until_complete
raise RuntimeError('Event loop stopped before Future completed.')
I am not sure where to investigate to debug it as the experiment fails only for larger matrices. The matrices are fixed-point arrays.
Moreover, a few months ago, I tried the same experiments with integer arrays and I could increase the number of columns up to 10K columns (before stopping the experiments to avoid memory overflow with larger matrices). Could the problem come from fixed-point arrays?
Looking forward to any input/intuition about this
Hi, @lschoe
I try to runmpyc
for some benchmark results on secure ID3 decision tree training.
I checked your paper "Practical Secure Decision Tree Learning in a Teletreatment Application", there are three algorithms that provide different security levels.
Currently, I'm not that familiar with the protocols in the paper
In mpyc
, it seems that you only implement SID3
.
Line 18 in 9876610
Do we implement SID3T
which provides a higher security level? I want to compare it with SID3T.
what are the encryption algorithm used for secint function
It seems that conversion from mpc.SecFld() to mpc.SecFxp() has some issues with fractional bits. In particular, the following code produces unexpected results:
from mpyc.runtime import mpc
async def main():
await mpc.start()
secfxp = mpc.SecFxp()
secfld = mpc.SecFld(min_order=2**10, signed=True)
a = secfxp(3)
a = mpc.input(a, 0)
print(await mpc.output(a))
b = mpc.convert(a, secfld)
print(await mpc.output(b))
c = mpc.convert(b, secfxp)
print(await mpc.output(c))
await mpc.shutdown()
if __name__ == "__main__":
mpc.run(main())
Result:
2020-03-03 17:01:02,303 Install package gmpy2 for better performance.
2020-03-03 17:01:02,305 Start MPyC runtime v0.6
3.0
3
196608.0
2020-03-03 17:01:02,320 Stop MPyC runtime -- elapsed time: 0:00:00.014997
Noting that, by default, SecFxp assumes 16 fractional bits, the fact that 3 * 2 ** 16 = 196608 suggests that there is some issue in handling the fractional bits. I've not yet had the time to look into this myself, but I can imagine that this issue is relatively easy to fix?
Hello, sorry for disturbing you.
I observed that when an array is created from np.zeros with mpc.SecInt and one is subtracted from it, it is still printed out as zeros instead of negative ones. In addition, when two is added to it, it is printed out as ones instead of twos.
Here is an example.
from mpyc.runtime import mpc
from mpyc.numpy import np
from mpyc import finfields
async def main():
f = mpc.SecInt(32)
a = np.array([[-1, -2, -3, -4], [0, 0, 0, 0], [1, 1, 1, 1], [1, 2, 3, 4]])
F_a = f.array(a)
print('F_a', await mpc.output(F_a))
F_b = f.array(np.zeros((4,4)))
F_b = F_b - 1
print('-1', await mpc.output(F_b))
F_b = F_b + 1
print('0', await mpc.output(F_b))
F_b = F_b + 1
print('1', await mpc.output(F_b))
mpc.run(main())
Here is the result.
F_a [[-1 -2 -3 -4]
[0 0 0 0]
[1 1 1 1]
[1 2 3 4]]
-1 [[0.0 0.0 0.0 0.0]
[0.0 0.0 0.0 0.0]
[0.0 0.0 0.0 0.0]
[0.0 0.0 0.0 0.0]]
0 [[0.0 0.0 0.0 0.0]
[0.0 0.0 0.0 0.0]
[0.0 0.0 0.0 0.0]
[0.0 0.0 0.0 0.0]]
1 [[1.0 1.0 1.0 1.0]
[1.0 1.0 1.0 1.0]
[1.0 1.0 1.0 1.0]
[1.0 1.0 1.0 1.0]]
When the zero array is created with finfields.GF, the array with negative ones can printed out successfully.
from mpyc.runtime import mpc
from mpyc.numpy import np
from mpyc import finfields
async def main():
f = finfields.GF(101)
a = np.array([[-1, -2, -3, -4], [0, 0, 0, 0], [1, 1, 1, 1], [1, 2, 3, 4]])
F_a = f.array(a)
print('F_a', await mpc.output(F_a))
F_b = f.array(np.zeros((4,4)))
F_b = F_b - 1
print('-1', await mpc.output(F_b))
F_b = F_b + 1
print('0', await mpc.output(F_b))
F_b = F_b + 1
print('1', await mpc.output(F_b))
mpc.run(main())
The result is shown below.
F_a [[-1 -2 -3 -4]
[0 0 0 0]
[1 1 1 1]
[1 2 3 4]]
-1 [[-1.0 -1.0 -1.0 -1.0]
[-1.0 -1.0 -1.0 -1.0]
[-1.0 -1.0 -1.0 -1.0]
[-1.0 -1.0 -1.0 -1.0]]
0 [[0.0 0.0 0.0 0.0]
[0.0 0.0 0.0 0.0]
[0.0 0.0 0.0 0.0]
[0.0 0.0 0.0 0.0]]
1 [[1.0 1.0 1.0 1.0]
[1.0 1.0 1.0 1.0]
[1.0 1.0 1.0 1.0]
[1.0 1.0 1.0 1.0]]
I cannot find the reason behind this behavior. Could you help me to explain this?
It would be very nice to have type hints implemented in MPyC. (Both for a better understanding of the code, as well as a more robust implementation)
Are there any plans to incorporate this in future versions?
Does MPyC have options of using multiple threads? I couldn't find it in the documentation.
Thanks!
For when is the new release for PyPI planned?
Hi @lschoe! I ran into an issue with integer division of secure integers. The following script fails for me when ran with at least three players (-M 3
):
# test.py
from mpyc.runtime import mpc
secint = mpc.SecInt()
async def main():
async with mpc:
a = mpc.input(secint(3), senders=0)
b = mpc.input(secint(2), senders=0)
print(await mpc.output(a // b)) # similarly, mpc.output(mpc.mod(a, b))
if __name__ == "__main__":
mpc.run(main())
$ python test.py -M3
2023-09-25 10:34:33,881 Start MPyC runtime v0.9
2023-09-25 10:34:34,189 All 3 parties connected.
2023-09-25 10:34:34,191 Unhandled exception in event loop
Traceback (most recent call last):
File ".../.pyenv/versions/3.11.1/lib/python3.11/asyncio/events.py", line 80, in _run
self._context.run(self._callback, *self._args)
File ".../.venv/lib/python3.11/site-packages/mpyc/asyncoro.py", line 431, in <lambda>
task.add_done_callback(lambda t: _reconcile(decl, t))
^^^^^^^^^^^^^^^^^^^
File ".../.venv/lib/python3.11/site-packages/mpyc/asyncoro.py", line 355, in _reconcile
givn = task.result()
^^^^^^^^^^^^^
File ".../.venv/lib/python3.11/site-packages/mpyc/asyncoro.py", line 283, in _wrap_in_coro
return await awaitable
^^^^^^^^^^^^^^^
File ".../.venv/lib/python3.11/site-packages/mpyc/asyncoro.py", line 271, in __await__
val = self.coro.send(None)
^^^^^^^^^^^^^^^^^^^^
File ".../.venv/lib/python3.11/site-packages/mpyc/runtime.py", line 530, in output
y = recombine(field, points)
^^^^^^^^^^^^^^^^^^^^^^^^
File ".../.venv/lib/python3.11/site-packages/mpyc/thresha.py", line 104, in recombine
s = share_i[h]
~~~~~~~^^^
IndexError: list index out of range
Traceback (enclosing MPyC coroutine call):
File ".../.venv/lib/python3.11/site-packages/mpyc/runtime.py", line 3206, in random_bits
r2s = await self.output(r2s, threshold=2*t)
Traceback (most recent call last):
File ".../test.py", line 14, in <module>
mpc.run(main())
File ".../.venv/lib/python3.11/site-packages/mpyc/runtime.py", line 181, in run
return self._loop.run_until_complete(f)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".../.pyenv/versions/3.11.1/lib/python3.11/asyncio/base_events.py", line 651, in run_until_complete
raise RuntimeError('Event loop stopped before Future completed.')
RuntimeError: Event loop stopped before Future completed.
I am aware that the docstring of SecureNumber.__floordiv__
states public divisor
. I wanted to work around this by using Runtime.mod
, which seems to be designed to work explicitly with secure divisors as it starts with b = await self.gather(b)
. Since __floordiv__
actually calls mod
under the hood, this yields the same error (but now the root of the issue is slightly narrowed down).
Should it indeed be possible to perform secure integer division/ modular reduction with public divisor/ modulus? If so, I am doing something wrong or this is a bug. If not, would you be willing to support (approximate) integer division?
Looking forward to your thoughts.
Dear lschoe,
I've been experiencing very strange MPyC behaviour since Monday. I run the following code with parameter "-M 3":
from mpyc.runtime import mpc
async def main():
async with mpc:
secnum = mpc.SecFxp()
a = secnum(2 if mpc.pid == 0 else None)
a = mpc.input(a, 0)
print(await mpc.output(a))
b = 1 / a
print(await mpc.output(b))
if __name__ == "__main__":
mpc.run(main())
Expected outcome:
2.0
0.5
However, the code results in the following error:
Traceback (most recent call last):
File "C:/Users/kamphorstb/git_repositories/CONVINCED/kaplan-meier/scripts/tester.py", line 15, in <module>
mpc.run(main())
File "C:\Users\kamphorstb\git_repositories\CONVINCED\kaplan-meier\.venv\lib\site-packages\mpyc\runtime.py", line 161, in run
return self._loop.run_until_complete(f)
File "C:\Users\kamphorstb\AppData\Local\Programs\Python\Python37\lib\asyncio\base_events.py", line 581, in run_until_complete
raise RuntimeError('Event loop stopped before Future completed.')
RuntimeError: Event loop stopped before Future completed.
2020-07-15 14:44:03,707 Start MPyC runtime v0.6.8
2020-07-15 14:44:05,048 All 3 parties connected.
2.0
2020-07-15 14:44:05,075 Exception in callback mpc_coro.<locals>.typed_asyncoro.<locals>.<lambda>(<Task finishe...ut of range')>) at C:\Users\kamphorstb\git_repositories\CONVINCED\kaplan-meier\.venv\lib\site-packages\mpyc\asyncoro.py:387
handle: <Handle mpc_coro.<locals>.typed_asyncoro.<locals>.<lambda>(<Task finishe...ut of range')>) at C:\Users\kamphorstb\git_repositories\CONVINCED\kaplan-meier\.venv\lib\site-packages\mpyc\asyncoro.py:387>
Traceback (most recent call last):
File "C:\Users\kamphorstb\AppData\Local\Programs\Python\Python37\lib\asyncio\events.py", line 88, in _run
self._context.run(self._callback, *self._args)
File "C:\Users\kamphorstb\git_repositories\CONVINCED\kaplan-meier\.venv\lib\site-packages\mpyc\asyncoro.py", line 387, in <lambda>
d.add_done_callback(lambda v: _reconcile(decl, v))
File "C:\Users\kamphorstb\git_repositories\CONVINCED\kaplan-meier\.venv\lib\site-packages\mpyc\asyncoro.py", line 299, in _reconcile
givn = givn.result()
File "C:\Users\kamphorstb\git_repositories\CONVINCED\kaplan-meier\.venv\lib\site-packages\mpyc\asyncoro.py", line 237, in _wrap
return await coro
File "C:\Users\kamphorstb\git_repositories\CONVINCED\kaplan-meier\.venv\lib\site-packages\mpyc\asyncoro.py", line 227, in __await__
val = self.coro.send(None)
File "C:\Users\kamphorstb\git_repositories\CONVINCED\kaplan-meier\.venv\lib\site-packages\mpyc\runtime.py", line 393, in _recombine
return thresha.recombine(field, points)
File "C:\Users\kamphorstb\git_repositories\CONVINCED\kaplan-meier\.venv\lib\site-packages\mpyc\thresha.py", line 79, in recombine
s = shares[i][h]
IndexError: list index out of range
2020-07-15 14:44:05,085 Task was destroyed but it is pending!
task: <Task pending coro=<_wrap() running at C:\Users\kamphorstb\git_repositories\CONVINCED\kaplan-meier\.venv\lib\site-packages\mpyc\asyncoro.py:237> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x000002E001D386A8>()]> cb=[mpc_coro.<locals>.typed_asyncoro.<locals>.<lambda>() at C:\Users\kamphorstb\git_repositories\CONVINCED\kaplan-meier\.venv\lib\site-packages\mpyc\asyncoro.py:387]>
2020-07-15 14:44:05,085 Task was destroyed but it is pending!
Running the same code in three terminals (additional parameter "-I x", x = 0, 1, 2) results in similar messages for players 0 and 1. Player 2 instead shows the following:
2020-07-15 14:30:25,173 Start MPyC runtime v0.6.8
2020-07-15 14:30:25,574 All 3 parties connected.
2.0
a: <mpyc.sectypes.SecFxp32:16 object at 0x000001753828C548>
f: 16
2020-07-15 14:30:26,047 Exception in callback _SelectorSocketTransport._call_connection_lost(ConnectionRes..., 10054, None))
handle: <Handle _SelectorSocketTransport._call_connection_lost(ConnectionRes..., 10054, None))>
Traceback (most recent call last):
File "C:\Users\kamphorstb\AppData\Local\Programs\Python\Python37\lib\asyncio\events.py", line 88, in _run
self._context.run(self._callback, *self._args)
File "C:\Users\kamphorstb\AppData\Local\Programs\Python\Python37\lib\asyncio\selector_events.py", line 926, in _call_connection_lost
super()._call_connection_lost(exc)
File "C:\Users\kamphorstb\AppData\Local\Programs\Python\Python37\lib\asyncio\selector_events.py", line 700, in _call_connection_lost
self._protocol.connection_lost(exc)
File "C:\Users\kamphorstb\git_repositories\CONVINCED\kaplan-meier\.venv\lib\site-packages\mpyc\asyncoro.py", line 125, in connection_lost
raise exc
File "C:\Users\kamphorstb\AppData\Local\Programs\Python\Python37\lib\asyncio\selector_events.py", line 814, in _read_ready__data_received
data = self._sock.recv(self.max_size)
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host
2020-07-15 14:30:28,334 Exception in callback _SelectorSocketTransport._call_connection_lost(ConnectionRes..., 10054, None))
handle: <Handle _SelectorSocketTransport._call_connection_lost(ConnectionRes..., 10054, None))>
Traceback (most recent call last):
File "C:\Users\kamphorstb\AppData\Local\Programs\Python\Python37\lib\asyncio\events.py", line 88, in _run
self._context.run(self._callback, *self._args)
File "C:\Users\kamphorstb\AppData\Local\Programs\Python\Python37\lib\asyncio\selector_events.py", line 926, in _call_connection_lost
super()._call_connection_lost(exc)
File "C:\Users\kamphorstb\AppData\Local\Programs\Python\Python37\lib\asyncio\selector_events.py", line 700, in _call_connection_lost
self._protocol.connection_lost(exc)
File "C:\Users\kamphorstb\git_repositories\CONVINCED\kaplan-meier\.venv\lib\site-packages\mpyc\asyncoro.py", line 125, in connection_lost
raise exc
File "C:\Users\kamphorstb\AppData\Local\Programs\Python\Python37\lib\asyncio\selector_events.py", line 814, in _read_ready__data_received
data = self._sock.recv(self.max_size)
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host
Here, the a: <mpyc.sectypes.SecFxp32:16 object at 0x000001753828C548>
and f: 16
are print statements that I put in mpc._rec()
. Players 0 and 1 did not print anything here.
The issue rises in MPyC versions 0.6, 0.6.5, 0.6.7, 0.6.8 (versions where this used to work!) and for various versions of Python 3. It also arises in a Docker environment (kind of ruling out firewall issues). My colleague experiences the same issue on a different workstation.
My colleague later noted that an easier example (which we did not quite investigate as thoroughly) kept running forever:
from mpyc.runtime import mpc
async def main():
async with mpc:
secnum = mpc.SecFxp()
a = secnum(2 if mpc.pid == 0 else None)
a = mpc.input(a, 0)
print(await mpc.output(a))
b = a * a # simpler than 1 / a
print(await mpc.output(b))
if __name__ == "__main__":
mpc.run(main())
I hope that you can reproduce the issue. If you need any further information I'd be happy to provide that.
Hi, I am going through the examples, and many examples I have looked at skip the step of "each party holds a secret file, and they concatenate it together", and assume that this step is done. (for example, the id3gini.py).
My question is on how this step is done, exactly. Is there an example that shows this in action? It can be very simple, like just concatenating files and output the total number of lines in the result file.
Hi,
I was wondering which protocol MPyC uses for truncation in the mpyc/runtime.py, since I cannot seem to find it in the sourcecode itself.
Thanks in advance!
Hi,
While implementing my own experiments, I've developed several utility functions that may interest other people here. For example, I implemented an np_shuffle
function that parallelizes the existing function shuffle
.
Hence, I wondered whether you had a documentation detailing contribution guidelines. In particular, I would like to know:
mpc.mul
gets stuck on SecFxp
inputs for more than 2 parties.
mpc.div
only works for SecFxp
inputs with 2 parties. SecInt
inputs will give wrong answers. SecFxp
inputs with more than 2 parties will result in errors.
mpc.reciprocal
seems to only work correctly for SecFld
Example code:
from mpyc.runtime import mpc
mpc.run(mpc.start())
secint = mpc.SecInt()
secfxp = mpc.SecFxp(32, 16)
secfld = mpc.SecFld(2**5)
a = mpc.input(secint(3), senders=0) if mpc.pid==0 else mpc.input(secint(None), senders=0)
b = mpc.input(secint(5), senders=1) if mpc.pid==1 else mpc.input(secint(None), senders=1)
c = mpc.input(secfxp(10), senders=0) if mpc.pid==0 else mpc.input(secfxp(None), senders=0)
d = mpc.input(secfxp(200), senders=1) if mpc.pid==1 else mpc.input(secfxp(None), senders=1)
e = mpc.input(secfld(9), senders=0) if mpc.pid==0 else mpc.input(secfld(None), senders=0)
u1 = mpc.mul(a, b) #mpc.mul(c, d) will get stuck if run with command-line argument -M3
u2 = mpc.div(c, d) #will fail if run with command-line argument -M3
u3 = mpc.reciprocal(e) #mpc.reciprocal(c) will give negative number
u4 = mpc.schur_prod([c], [d])[0] #work-around for mpc.mul(c,d)
o1 = mpc.run(mpc.output(u1))
o2 = mpc.run(mpc.output(u2))
o3 = mpc.run(mpc.output(u3))
o4 = mpc.run(mpc.output(u4))
print('output:', o1, o2, o3, o4)
mpc.run(mpc.shutdown())
Is it possible to "hardcode" the number of local parties, e.g. 3, ($ mpyc_script.py -M3) so that i can run a function of that script (mpyc_script.make_shares(arguments)) without having to run the complete script (os.system("python mpyc_script.py -M3"))?
thx
I want to perform MPC between two parties to compute encryption over two blocks of messages. I want one party to supply the key as private input and the other party to supply the blocks. Is there an easy way to go about this? I checked the aes demo but I'm not super clear.
I wrote the following program, modmath.py:
import numpy as np
from mpyc.runtime import mpc
async def main():
await mpc.start()
secint19 = mpc.SecFld(2**255 - 19)
a = secint19(10)
a_bits = mpc.to_bits(a, 6)
a_bits = await mpc.output(a_bits)
print("a_bits=", str(a_bits))
await mpc.shutdown()
if __name__ == '__main__':
mpc.run(main())
I ran it:
python3 modmath.py -M 3
I expect when I run it I would get the output
a_bits=[0, 1, 0, 0, 1, 0]
But the output is non-deterministic, with the above format but a seemingly random choice of zeros and ones.
Is this a bug or am I misunderstanding something?
Hi, a question puzzle me when i read the code in runtime.py, about the function mul(self, a, b), why gather(a)๏ผ gather operation will reveal the true value of a to the peer, i think this operation is not secure. is my understanding wrong?
In the output method of runtime.py, the output of SecFlt uses the SecureFloat._output method. However, the SecureFloat._output method recursively calls the output method in runtime.py. This can lead to issues where, if the current node is not in the receivers, x_s and x_e become None, resulting in assertion errors and errors in the return values in the SecureFloat._output method.
I hope to find a solution for this issue. If my understanding is incorrect, please let me know.
Line 32 in e04f5b6
# polynomial f(x) = s[h] + c[0] x + c[1] x^2 + ... + c[d-1] x^d
for i in range(n):
y = 0
for c_k in c:
y += c_k
y *= i + 1
Would the code not compute f(x) = s[h] + c[0] x^d + ... c[d-2] x^2 + c[d-1] x^1
Uncertain whether this is a real issue, and if it is what the best fix would be.
Hi, I am doing some experiments using MPyC and I am stuck on a little problem: I would like to sort a list of tuples but I have an error. I don't understand yet all the details of the library so I would like your insights before implementing my sorting procedure manually.
Here is a minimal example of the problem:
from mpyc.runtime import mpc
from mpyc.seclists import seclist
def example_tuple_sort():
secint = mpc.SecInt(64)
res = seclist([], sectype=seclist)
res.append(seclist([secint(1), secint(2)], sectype=secint))
res.append(seclist([secint(0), secint(4)], sectype=secint))
res.sort(key=lambda tup: tup[1])
If you try to run this function, you obtain the following error:
<ipython-input-235-ca59bed29c48> in example_tuple_sort()
4 res.append(seclist([secint(1), secint(2)], sectype=secint))
5 res.append(seclist([secint(0), secint(4)], sectype=secint))
----> 6 res.sort(key=lambda tup: tup[1])
7
~/.local/lib/python3.10/site-packages/mpyc/seclists.py in sort(self, key, reverse)
331 if key is None:
332 key = lambda a: a
--> 333 runtime._sort(self, key)
334 if reverse:
335 self.reverse()
~/.local/lib/python3.10/site-packages/mpyc/runtime.py in _sort(self, x, key)
1113 if i & p == r:
1114 a, b = x[i], x[i + d]
-> 1115 x[i], x[i + d] = self.if_swap(key(a) >= key(b), a, b)
1116 d, q, r = q - p, q >> 1, p
1117 p >>= 1
~/.local/lib/python3.10/site-packages/mpyc/runtime.py in if_swap(self, c, x, y)
1641
1642 if isinstance(x, list):
-> 1643 z = self._if_swap_list(c, x, y)
1644 else:
1645 d = c * (y - x)
~/.local/lib/python3.10/site-packages/mpyc/asyncoro.py in typed_asyncoro(*args, **kwargs)
384 while True:
385 try:
--> 386 coro.send(None)
387 except StopIteration as exc:
388 runtime._pc_level -= 1
~/.local/lib/python3.10/site-packages/mpyc/runtime.py in _if_swap_list(self, a, x, y)
1622 await self.returnType((stype, x[0].integral and y[0].integral), 2, n)
1623
-> 1624 a, x, y = await self.gather(a, x, y)
1625 if f:
1626 a = a >> f # NB: no in-place rshift!
~/.local/lib/python3.10/site-packages/mpyc/runtime.py in gather(self, *obj)
146
147 def gather(self, *obj):
--> 148 return asyncoro.gather_shares(self, *obj)
149
150 async def barrier(self, name=None):
~/.local/lib/python3.10/site-packages/mpyc/asyncoro.py in gather_shares(rt, *obj)
223 return _SharesCounter(rt._loop, obj)
224
--> 225 return _AwaitableFuture(_get_results(obj))
226
227
~/.local/lib/python3.10/site-packages/mpyc/asyncoro.py in _get_results(obj)
198
199 if isinstance(obj, (list, tuple)):
--> 200 return type(obj)(map(_get_results, obj))
201
202 return obj
~/.local/lib/python3.10/site-packages/mpyc/asyncoro.py in _get_results(obj)
198
199 if isinstance(obj, (list, tuple)):
--> 200 return type(obj)(map(_get_results, obj))
201
202 return obj
~/.local/lib/python3.10/site-packages/mpyc/seclists.py in __init__(self, x, sectype)
60
61 if sectype is None:
---> 62 raise ValueError('sectype missing')
63
64 i = 0
If I understand well, it seems like the sorting procedure swaps the lists and re-instantiates the seclist
object, but the parameter sectype is not provided which causes the error. By the way, if you give me some instructions, I can contribute to the repo and fix this potential bug (I am not sure whether the problem is a bug or my code for now).
Thank you for your help and thank you for this great library! :)
Thank you for your excellent work!
I have been using your library for a while. I am now trying to use the "lpsolver.py" for a particular dataset. However, I am unsure how to set the correct "bit-length." Can you elaborate more, please?
Best,
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.