alexandredecan / portion Goto Github PK

First of all, thank you so much for the incredibly useful library!

I'd like to know if it is possible to obtain the interval which was used to store a value in an IntervalDict when querying.

For example, if I store something with d[P.closed(0, 10)] = 'whatever' and then query the dict for d.get(P.singleton(5)), is it possible for me to know that the interval in the dict that contains the interval I used for querying is P.closed(0, 10)?

Thanks again

now I need to get some random values from a interval instance, is there some simple methods to achieve this?
if there is, I would appreciate it very much !

Here's a gist showing some of the bugs.

As a continuation of #49 :),

I'm now trying to read back from two IntervalDicts, compare the values assigned to each interval, modify the values, and write this all back to files.

[As a follow-up to #49, I have anywhere between 1-18700 intervals per value (with ~2800 values), with average size being ~216 intervals per value. (Did not collect the median, so no idea about the actual distribution 🙂 ) ]

I am now trying to iterate over all points (sequentially or not doesn't really matter as long as I check all the points) with this code (sorry for verbosity, I like seeing it do stuff):

# interval1 and interval2 are IntervalDict instances
for key in tqdm(interval1.keys(), desc="Key", position=0):
        tqdm.write(f"key: {str(key):.30}", file=sys.stderr)
        tqdm.write(f"key size: {str(len(key))}", file=sys.stderr)  # 17500 for the first element!
        for position in tqdm(P.iterate(key, step=1), desc="Position", position=1):
            # tqdm.write("pos: {:.30}".format(str(position)), file=sys.stderr)  # first value is 119
            c1 = interval1[position]
            c2 = interval2[position]
            if c2 > 0 and float(c1) / c2 >= threshold:
                # change c1 and c2 here
                filtered_counter += 1
            elif c1 > 0 and float(c2) / c1 >= threshold:
                # change c1 and c2 here
                filtered_counter += 1
            # creating plain dicts of modified values first, will then convert to IntervalDict - faster this way
            new_interval1_dict[c1].append(P.singleton(position))
            new_interval2_dict[c2].append(P.singleton(position))
    # TODO: convert into IntervalDict and return

The speed looks very variable, mostly in the 6-12 positions/sec range, but I've also seen peak values of 98 pos/sec.
So far 13k position processed in 15 minutes; if the speed sample is representative, this will take 26 hours.

@AlexandreDecan , you mentioned a smarter way of iterating, can you please share it? 🙂

I don't know if it was intended but when I update a Portion IntrervalDict with an interval that overlaps another already contained interval both the items remain. It behaves as a normal dict for some reason.

Hello!

We are using python intervals for our current project, and it's fits very well to our needs. I noticed yesterday that the interval inversion operation is very slow at medium scales. I have compound intervals of datetimes of the scale of hundreds of atomic intervals, and on a reasonably recent computer, the inversion of such intervals takes seconds to complete, which I find strange because I wouldn't say that kind of computation would be that heavy. Here below is some data on the processing time I'm observing. Maybe that reveals some possible optimizations for this operator.

┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓
┃ Nb Intervals ┃ Processing Time     ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩
│ 92           │ 0.13                │
│ 92           │ 0.12                │
│ 92           │ 0.12                │
│ 801          │ 7.42                │
│ 1396         │ 22.70               │
│ 1185         │ 16.29               │
│ 639          │ 4.76                │
│ 1029         │ 12.98               │
│ 1662         │ 35.29               │
│ 2266         │ 67.45               │
│ 891          │ 10.37               │
│ 1052         │ 14.55               │
│ 640          │ 5.34                │
│ 1316         │ 22.73               │
│ 762          │ 7.56                │
│ 870          │ 9.77                │
│ 1187         │ 18.50               │
│ 1002         │ 13.06               │
│ 921          │ 10.63               │
│ 650          │ 5.22                │
│ 861          │ 9.07                │
│ 1585         │ 30.88               │
│ 926          │ 10.45               │
│ 1599         │ 31.22               │
│ 1226         │ 18.58               │
│ 663          │ 5.87                │
│ 218          │ 0.58                │
│ 430          │ 2.27                │
│ 1527         │ 31.98               │
│ 92           │ 0.10                │
│ 92           │ 0.10                │
│ 92           │ 0.10                │
│ 1581         │ 30.89               │
│ 1988         │ 47.65               │
└──────────────┴─────────────────────┘

Hello,

I was wondering if it will be possible to add a function that could say if an interval is a singleton or not ?
For the moment I just did i.lower==i.upper but I found it dirty. A solution with a function is_singleton will be more elegant.

Thank you in advance

I am trying to import roughly 600k intervals from a text file like this, with start/end/value columns, where values are relatively repetitive, and start-end pairs together form a single long contiguous interval (no gaps):

0       119     0
119     120     1
120     130     4
130     133     0
...

Right now I am iterating over lines of the file, doing intervals[P.closedopen(int(start), int(end))] = int(value) on each parsed line.
This process becomes progressively slower and slower, dropping to just 20 lines/second after ~6500 lines.
Upon closer examination, IntervalDict seems to simplify intervals upon each addition in such a way that same-value disjoint intervals are grouped together:

IntervalDict.domain of {[119,120) | [375,450) | [671,672) | [705,706) | [870,872) | 
[2271,2276) | [2351,2363) | [2385,2400) | [2477,2503) | [2541,2548) | [2578,2582) | 
[2786,2788) | [3698,3702) | [3706,3756) | [3831,3850) | [3851,3857): 1, [120,130) | 
[292,295) | [586,587) | [880,889) | [1268,1278) | [1306,1321) | [1327,1328) | 
[1381,1396) | [1925,1932) | [2007,2008) | [2018,2023) | [2106,2117) | [2135,2136) |
[2138,2153) | [2196,2213) | [2762,2778) | [2793,2795) | [2891,2898) | [2988,2992) | 
[3114,3122) | [3688,3695) | [3894,3907) | [3925,3932): 4,

I believe this is the reason for a progressive slowdown.

My actual questions are:

• is there a more efficient way to populate IntervalDict? for example, by populating a dict first, and then "converting" into IntervalDict?
• is it possible to disable the merging functionality while adding intervals, and then call it once at the end?

Intervals can be provided to build new intervals. The constructor expects to receive a list of intervals or atomic intervals. If two empty atomic intervals are provided, the following assumption holds:

assert I.Interval(I.empty().to_atomic(), I.empty().to_atomic()) == I.empty()

but if two empty non-atomic intervals (i.e., non AtomicInterval instances) are provided, the equivalent assumption does not hold:

assert I.Interval(I.empty(), I.empty()) == I.empty()

The reason is that we check for emptiness only for AtomicInterval instances, and not for Interval ones.

Very easy to reproduce:

a = portion.closed(date(2020, 2, 1), date(2020, 2, 28)) | portion.closed(date(2020, 3, 1), date(2020, 3, 31))
b = portion.closed(date(2020, 4, 1), date(2020, 4, 30))
print(b in a)

With numbers it's the same:

a = portion.closed(1, 2) | portion.closed(3, 4)
b = portion.closed(5, 6)
print(b in a)

Exception:

File "venv/lib/python3.8/site-packages/portion/interval.py", line 478, in __contains__
    current = next(selfiter)
StopIteration

Is it not supposed to work this way? Or is it a bug?

Thanks!

(From #13, @msarrel):

One feature that would be useful for me is to define addition, subtraction, etc. for IntervalDict. That is if we have two IntervalDicts:

[1, 3] | [5, 7] = 1
and
[2, 4] | [6, 8] = 2

Their sum would be a single Interval Dict with three entries:

[1, 2) | [5, 6) = 1
[2, 3) | [6, 7) = 3
[3, 4] | [7, 8] = 2

The idea would be to provide a d.combine(other_d, func) that takes values from d and other_d and applies func on them for every intersecting pair of intervals (as in the example, letting the values of the non-intersecting parts of these intervals as-is).

Unioning AtomicInterval instances (and by extension Interval instances) can lead to an invalid result if the two intervals share a (lower or upper) bound, where one is open and the other one is closed.

>>> I.closed(0, 2) | I.open(0, 2)
I.open(0, 2)

... where the expected result is I.closed(0, 2).

2.0.0 is still under development, but we already observed a huge performance increase for interval creation/union, intersection, difference and complement (except for intersection of very small intervals, but we are talking about µs!).

TL;DR: speed-up observed, details below.

                                        Union  Intersection  Difference  Complement
large intervals: ~500 atomics        1.854881     36.768802  117.073171  234.741784
medium intervals: ~50 atomics        1.637532      4.327869   12.640000  349.353050
small intervals: ~5 atomics          1.349882      0.463259    1.505017    3.898305

Time in milliseconds (5K atomics not reported for 1.10.0: too slow):

                                          v     Union  Intersection  Difference  Complement
index                                                                                      
small intervals: ~5 atomics          1.10.0   0.00571        0.0145      0.0450     0.02990
medium intervals: ~50 atomics        1.10.0   0.06370       13.2000     31.6000    18.90000
large intervals: ~500 atomics        1.10.0   7.03000     1320.0000   2880.0000  1500.00000
small intervals: ~5 atomics           2.0.0   0.00423        0.0313      0.0299     0.00767
medium intervals: ~50 atomics         2.0.0   0.03890        3.0500      2.5000     0.05410
large intervals: ~500 atomics         2.0.0   3.79000       35.9000     24.6000     6.39000
very large intervals: ~5000 atomics   2.0.0  69.20000      258.0000    205.0000    50.50000

To quantify this, we ran the following snippet in both versions (in a Jupyter console, hence the use of %timeit). Please note that there is no scientific or rigorous process here ;-)

import intervals as I

sm = [
    I.Interval(*(I.closed(i, i+1) for i in range(0, 10, 2))),
    I.Interval(*(I.closed(i, i+1) for i in range(1, 10, 2)))
]

md = [
    I.Interval(*(I.closed(i, i+1) for i in range(0, 100, 2))),
    I.Interval(*(I.closed(i, i+1) for i in range(1, 100, 2)))
]

xl = [
    I.Interval(*(I.closed(i, i+1) for i in range(0, 1000, 2))),
    I.Interval(*(I.closed(i, i+1) for i in range(1, 1000, 2)))
]

xxl = [
    I.Interval(*(I.closed(i, i+1) for i in range(0, 10000, 2))),
    I.Interval(*(I.closed(i, i+1) for i in range(1, 10000, 2)))
]


for i, label in zip([sm, md, xl, xxl], ['small', 'medium', 'large', 'very large']):
    print(label, 'intervals: ~'+str(len(i[0])), 'atomics')
    print('Union:')
    %timeit -r5 -n10 I.Interval(*i)

    print('Intersection:')
    %timeit -r5 -n10 i[0] & i[1]

    print('Difference:')
    %timeit -r5 -n10 i[0] - i[1]

    print('Complement:')
    %timeit -r5 -n10 ~i[0]

    print()

Here are the resulting outputs for both versions. Notice that we killed the execution of "very large intervals" in 1.10.0 as it took too much time.

1.10.0 :

small intervals: ~5 atomics
Union:
57.1 µs ± 3.14 µs per loop (mean ± std. dev. of 5 runs, 10 loops each)
Intersection:
145 µs ± 19.6 µs per loop (mean ± std. dev. of 5 runs, 10 loops each)
Difference:
450 µs ± 26.1 µs per loop (mean ± std. dev. of 5 runs, 10 loops each)
Complement:
299 µs ± 34.9 µs per loop (mean ± std. dev. of 5 runs, 10 loops each)

medium intervals: ~50 atomics
Union:
637 µs ± 36.2 µs per loop (mean ± std. dev. of 5 runs, 10 loops each)
Intersection:
13.2 ms ± 777 µs per loop (mean ± std. dev. of 5 runs, 10 loops each)
Difference:
31.6 ms ± 4.05 ms per loop (mean ± std. dev. of 5 runs, 10 loops each)
Complement:
18.9 ms ± 2.46 ms per loop (mean ± std. dev. of 5 runs, 10 loops each)

large intervals: ~500 atomics
Union:
7.03 ms ± 577 µs per loop (mean ± std. dev. of 5 runs, 10 loops each)
Intersection:
1.32 s ± 53.7 ms per loop (mean ± std. dev. of 5 runs, 10 loops each)
Difference:
2.88 s ± 53.3 ms per loop (mean ± std. dev. of 5 runs, 10 loops each)
Complement:
1.5 s ± 37.3 ms per loop (mean ± std. dev. of 5 runs, 10 loops each)

2.0.0 :

small intervals: ~5 atomics
Union:
42.3 µs ± 1.07 µs per loop (mean ± std. dev. of 5 runs, 10 loops each)
Intersection:
313 µs ± 9.13 µs per loop (mean ± std. dev. of 5 runs, 10 loops each)
Difference:
299 µs ± 15.1 µs per loop (mean ± std. dev. of 5 runs, 10 loops each)
Complement:
76.7 µs ± 1.34 µs per loop (mean ± std. dev. of 5 runs, 10 loops each)

medium intervals: ~50 atomics
Union:
389 µs ± 19.5 µs per loop (mean ± std. dev. of 5 runs, 10 loops each)
Intersection:
3.05 ms ± 128 µs per loop (mean ± std. dev. of 5 runs, 10 loops each)
Difference:
2.5 ms ± 308 µs per loop (mean ± std. dev. of 5 runs, 10 loops each)
Complement:
541 µs ± 15.4 µs per loop (mean ± std. dev. of 5 runs, 10 loops each)

large intervals: ~500 atomics
Union:
3.79 ms ± 202 µs per loop (mean ± std. dev. of 5 runs, 10 loops each)
Intersection:
35.9 ms ± 4.81 ms per loop (mean ± std. dev. of 5 runs, 10 loops each)
Difference:
24.6 ms ± 3.56 ms per loop (mean ± std. dev. of 5 runs, 10 loops each)
Complement:
6.39 ms ± 1.11 ms per loop (mean ± std. dev. of 5 runs, 10 loops each)

very large intervals: ~5000 atomics
Union:
69.2 ms ± 10.3 ms per loop (mean ± std. dev. of 5 runs, 10 loops each)
Intersection:
258 ms ± 11.3 ms per loop (mean ± std. dev. of 5 runs, 10 loops each)
Difference:
205 ms ± 10.7 ms per loop (mean ± std. dev. of 5 runs, 10 loops each)
Complement:
50.5 ms ± 3.75 ms per loop (mean ± std. dev. of 5 runs, 10 loops each)

Dear author，

Thank you for your excellent work! I have a question:

when the interval is not atomic, eg,

x = P.open(0, 1) | P.closed(3, 4)

I can use x.lower and x.upper get 0 and 4. Which attribute can I use to access 1 and 3?

Here is what I mean:

Rather than this:

def __or__(self, other):
    if isinstance(other, Interval):
        return Interval(self, other)
    else:
        return NotImplemented

I think that something like this is preferable:

def __or__(self, other):
    if isinstance(other, self.__class__):
        return self.__class__(self, other)
    else:
        return NotImplemented

The reason for my feeling here is that it allows one to subclass your extremely helpful Interval class and allows that subclass to return instances of itself when making an or comparison, for example.

Lets say that I have two objects that are subclasses of Interval (SpecialInterval): a and b.

In your code, type(a | b) returns Interval.

I think most people would expect it to return a SpecialInterval object since we were comparing two SpecialInterval instances.

Does this make sense?

That's a regression introduced in 2.0.0. Non-atomic intervals composed of singletons are not properly displayed when they are repr-ed:

>>> P.singleton(1) | P.singleton(2)
[1,1] | [2,2]

It should be [1] | [2].

This has no practical incidence, and this regression does not affect from_string and to_string.

Hallo,

I have bumped the version of satmad, my library that depends on portion. Conda-forge process failed with the following error:

portion 2.1.3 has requirement sortedcontainers~=2.2.2, but you have sortedcontainers 2.3.0.

Looks like there is a hard dependency on a specific version of sortedcontainers. Is this strictly necessary? Is there any issue with a specific version of it?

Cheers!

Items in an IntervalDict are sorted according to their lower bound, but when a bound is shared by two intervals (= keys), the open interval is listed first instead of the closed one:

>>> d = P.IntervalDict()
>>> d[P.open(2, 3)] = 1
>>> d
{(2,3): 1}
>>> d[2] = 2
>>> d
{(2,3): 1, [2]: 2}

The expected output should be [2] then (2, 3).

Hi,

Thank you for making all the improvements in your library in portion 2. However, one of your changes from python-intervals to portion broke my code.

In python-intervals, if I passed an Interval to the IntervalDict.get() method, that method would return the value if the Interval corresponded to a single value in the IntervalDict. In portion, now the get() method always returns a new IntervalDict, even if there is only one entry. I would like a way to have the old functionality as an option, while also keeping the new functionality.

The Python dict.get() method always returns the value, so I would maybe suggest that IntervalDict.get() follow that precedent, and throw an exception if there is more than value for the submitted Interval. Then, the new functionality could be provided by a new method, or by setting a flag to True in the existing IntervalDict.get() method.

Here is an example of my use case. My goal is to iterate over the values in the IntervalDict, one for single-valued atomic interval, in sorted order. I have a cheap work-around, but it fails for infinite intervals.

import portion

d = portion.IntervalDict()
d[portion.closed(0, 3)] = 'banana'
d[4] = 'apple'
d[portion.closed(5, 8)] = 'orange'
d[portion.closed(9, 12)] = 'banana'
d[portion.closed(13, portion.inf)] = 'pomegranate'

print()
print('# The interval dictionary')
print(d)

print()
print('# Sorted list of atomic intervals')
print(list(d.domain()))

print()
print('# I want the value, not another IntervalDict')
print('# This worked in python-intervals, but fails in portion')
for i in list(d.domain()):
    print(d.get(i))

print()
print('# This work-around produces the desired result')
print('# But it fails for infinite intervals')
for i in list(d.domain()):
    print(d.get((i.lower + i.upper)/2))

Here is the output I get when I run the above example.

# The interval dictionary
{[0,3] | [9,12]: 'banana', [4]: 'apple', [5,8]: 'orange', [13,+inf): 'pomegranate'}

# Sorted list of atomic intervals
[[0,3], [4], [5,8], [9,12], [13,+inf)]

# I want the value, not another IntervalDict
# This worked in python-intervals, but fails in portion
{[0,3]: 'banana'}
{[4]: 'apple'}
{[5,8]: 'orange'}
{[9,12]: 'banana'}
{[13,+inf): 'pomegranate'}

# This work-around produces the desired result
# But it fails for infinite intervals
banana
apple
orange
banana
Traceback (most recent call last):
  File "<input>", line 28, in <module>
TypeError: unsupported operand type(s) for +: 'int' and '_PInf'

Hi , your application is working good and no issue .

But i have another problem like ,

• Actually its working for single type interval for ex:
  (c1>10 and c1 <20) or (c1 >30 and c1 <50)
  So this one i am changing to Interval format and evaluating. and getting information as
  c1 lower value is 10
  C1 upper value is 50

now i need in next level as instead of 1 if multiple are there how to do:
for ex:
(c1>10 and c1 <20) or (c1 >30 and c2 <50)
so i am expecting
c1 .lower and upper is (10 to +inf)
c2 lower and upper is (-inf to +inf)

This is not an issue , i need is there any way i can handle this.

Hi,
First off all, an amazing thank you for this library

I would like to subtract two intervals of panda's timestamp.
But it seems to crash when it tries to compare infinite value with a timestamp

a = I.closed(Timestamp('2018-11-11 09:00:00'),Timestamp('2018-11-11 17:00:00'))
b = I.closed(Timestamp('2018-11-11 10:00:00'),Timestamp('2018-11-11 12:00:00'))
c = a - b

Here is the traceback

File "\lib\site-packages\intervals.py", line 753, in __sub__
    return self & ~other
File "\lib\site-packages\intervals.py", line 745, in __invert__
    complements = [~i for i in self._intervals]
File "\lib\site-packages\intervals.py", line 745, in <listcomp>
    complements = [~i for i in self._intervals]
File "\lib\site-packages\intervals.py", line 432, in __invert__
    AtomicInterval(inverted_right, self._upper, inf, OPEN)
File "\lib\site-packages\intervals.py", line 242, in __init__
    if self.is_empty():
File "\lib\site-packages\intervals.py", line 283, in is_empty
    self._lower > self._upper or
File "pandas\_libs\tslibs\timestamps.pyx", line 170, in pandas._libs.tslibs.timestamps._Timestamp.__richcmp__
TypeError: Cannot compare type 'Timestamp' with type '_PInf'

Thank you in advance

As it is now we have to instantiate I.empty() to compare if the interval is empty:

i1 = I.empty()
i1 == I.empty()
True
i1 is I.empty()
False

In certain applications this can have a significant performance impact. Would it be possible to have a singleton for an empty interval, which would allow checking using "is" operator?

.overlaps working wrong when intervals are contiguous
examples:

I.closed(1, 2).overlaps(I.closed(2, 3))
True
I.closed(1, 2).overlaps(I.closedopen(2, 3))
False (should be True)
I.closed(1, 2).overlaps(I.closed(2, I.inf))
False (should be True)

I.closed(2, I.inf).overlaps(I.closed(1,2))
False (should be True)
I.closed(2, 3).overlaps(I.closed(1,2))
True
I.closedopen(2, 3).overlaps(I.closed(1,2))
False (should be True)

I.closed(datetime.date(2020,10,18),I.inf) & I.closed(datetime.date(2020,11,29),I.inf)
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib/python2.7/site-packages/intervals.py", line 101, in closed
return Interval(AtomicInterval(CLOSED, lower, upper, CLOSED))
File "/usr/lib/python2.7/site-packages/intervals.py", line 382, in init
if self.is_empty():
File "/usr/lib/python2.7/site-packages/intervals.py", line 423, in is_empty
self._lower > self._upper or
TypeError: can't compare datetime.date to instance

Are you sure there is this method?
I have "x in y" but not "x.contains(y)"

should be updated in the docs too

intervals.py:538: RuntimeWarning: invalid value encountered in less_equal
left = (item >= self._lower) if self._left == CLOSED else (item > self._lower)

RuntimeWarning: invalid value encountered in greater
right = (item <= self._upper) if self._right == CLOSED else (item < self._upper)

Anaconda3 2019.7
win10_1903 x64

Hullo,

I have been trying to upload my package to conda-forge and it turned out that I can't, as portion is not there and conda-forge really does not want external (read: pypi-only) dependencies.

My question is, would you consider uploading portion to conda-forge so that I can refer to your package there as a dependency?

Hi; thanks for that great library!

While using portion.IntervalDict, I noticed that it is not fully respecting the interface of the dict class: the .keys(), .values() and .items() methods should return view objects instead of list objects. It would be really helpful to have this mechanism implemented (it would slightly improve performances as well in some specific circumstances).

Best.

Python 3.9 will be released on October 5th. Among the chances for this version, dict now supports | and |=, respectively for merge and update. See https://www.python.org/dev/peps/pep-0584/

Since IntervalDict aims to follow the interface of a traditional dict, we should implement them as well.
|= will be easy to do, since we can simply delegate to IntervalDict.update.
For |, it is a bit more tricky, and the current plan would be to first copy the current instance, then update this copy and return it.

Since that's a quite easy issue to address, let's label it as "good first issue" ;)

Other than specified in the documentation it is impossible to set closed interval bounds explicitly with replace (at least I can't). When an interval boundary is closed, it remains closed unless you want to set it explicitly closed.
Example:

>>> i = P.closedopen(0, 2)
>>> i.replace()
[0,2)
>>> i.replace(left=P.CLOSED)
(0,2)
>>> i.replace(left=lambda v: P.CLOSED)
(0,2)
>>> i.replace(lower=1)
[1,2)

hi Alexandre Decan
I have been using your library and it has been really useful however, I want to know if there is a way to output an interval as an array of numbers?
for example :
print( I.closed(0, 5)) returns something like [0, 1, 2, 3, 4, 5]

import portion as p

print(p.closed(0, 1).contains(p.empty()))

The above prints True, meaning than any interval contains the empty interval. Is that intended?

You say that discrete intervals can be simulated though with portion. How would you recommend to simulate this?

The empty interval is not contained in non-atomic intervals:

>>> P.empty() in P.open(1,2) | P.open(3,4)
False

The reason should be within the function __contains__ of class Interval. In case that item == P.empty() also other is the empty interval and therefore current < other for every atomic interval current of self. Thus, when iterating selfiter, a StopIteration is thrown and the function returns False.

e.g.
P.open(1, 2).size() == 1
P.closed(1, 2).size() == 1
(P.open(1, 2) | P.closed(4, 5)).size() == 2

Hi Alexandre,

I love this library. It comes close to almost everything I would expect from a standard library for handling intervals. There's one feature I'm missing that I would love to see added or add myself, and that's a built-in way to add a value to all bounds of an interval (whether atomic or union).

I realize this can be done with .apply() (in fact, it's even one of the examples in the readme), but in the project where I would love to use portion it would be such a common occurrence it becomes really tempting to build in support for it, instead of handling it with lambda functions or an external wrapper for .apply.

In the planning software I'm working on we use timedelta intervals, tasks to be planned relative to some event time. Adding the event time to the timedelta interval would yield an interval of datetimes that represents the planned task. It seems to me that these and other use cases could be common enough to warrant a convenient function, like.offset(value) or .translate(value), that would be a shortcut for .apply(lambda x: (x.left, x.lower + value, x.upper + value, x.right)).

What do you think? Would you be open for a pull-request for it?

Best regards, Maarten

I am too writing an interval library for C#, and while searching for design inspiration, came across portion. So I tried it out (never ever used Python before) and got this: math.nan in (-inf, inf) == true. Is that a bug? Doesn't seem to be the case for (-inf, 0) or (0, inf).

Also, (math.nan, 0).empty == false. This definitely seems like a bug.

Empty intervals do not properly compare because of the lower and upper bounds I.inf:

>>> I.empty() in I.open(-I.inf, I.inf)
True
>>> I.empty() in I.closed(0, 1)
False

Even worse, if AtomicInterval instances are directly created with an empty interval, containment depends on the values, not on the "emptiness" of the interval:

>>> i = I.AtomicInterval(False, 0, 0, False)
>>> i in I.closed(-3, -2)
False
>>> i in I.closed(-3, 3)
True

Similar issue holds when intervals are compared:

>>> i < I.singleton(0)
True
>>> i < I.singleton(-1)
False

Empty intervals should always be considered as included in other intervals, and should always compare to true when compared to other intervals.

I use IntervalDict to store pandas DataFrame. Because of this line #255 setting element to IntervalDict fails - DataFrame can't be compared to just any other object.

from pandas import DataFrame

d = portion.IntervalDict()
d[portion.singleton(1)] = DataFrame()
d[portion.singleton(2)] = DataFrame()

exit()

File "df.py", line 89, in <module>
    d[portion.singleton(2)] = DataFrame()
  File "venv/lib/python3.8/site-packages/portion/dict.py", line 255, in __setitem__
    if value == v:
  File "venv/lib/python3.8/site-packages/pandas/core/generic.py", line 1478, in __nonzero__
    raise ValueError(
ValueError: The truth value of a DataFrame is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().

As a workaround I assign tuple (id(dataframe), dataframe), which prevents comparing DataFrame. But that's awkward.

Thanks!

Hi there - firstly, thank you for saving me days of work with this library <3 Hugely appreciated, and it's very well done.

In the following, no error is produced:

import portion as P

ival = P.from_string('[10, 20] | 25', int)
print(ival) #-> "[10,20]"

Would it be possible to produce an error if there is unexpected input? Or, alternatively, would it be possible to have a check_string(s) function that performed a full-string match based on the same parameters?

As of right now, the invalid | 25 gets swallowed.

Thanks for any information!

Quick question: Is insertion order preserved when querying from IntervalDict?

Very nice library - thanks!

I was wondering if you have a suggested method for chaining interval operations on lists of intervals?

For example for intersecting several lists you can use:

r = P.closed(100.1239812, 300.9012312) | P.closed(110, 300.9012312) | P.closed(97, 300.9012312)

If you have a list of unknown size is it possible to chain the intersection?

It would be nice if one of the approaches for chaining sets could be used as described here:

>> list(set.union(*map(set, a)))
[1, '44', '30', '42', '43', '40', '41', '34']

>>> from itertools import chain
>>> list(set(chain.from_iterable(d)))

So something like:

>>> from itertools import chain
>>> list(P.intersection(chain.from_iterable(d)))

Maybe this is already possible but I've not found a successful approach so far (beyond a simple loop and applying intersection one by one).

Hi

I was wondering if it will be possible to make Interval and AtomicInterval JSON serializable ? 🙏 😄

Thank you in advance

Nice library and it saved my great efforts to manipulate between intervals.
During my process of getting to learn about your library, I wrote down a Chinese version documentation (CSDN link, maybe naive, but enough to know basic usage and advanced operations) which attached to this comment.
Hope can be of little help to our Chinese users. Again, thanks for your nice library and hope more coders can know about it.

portion-doc-zh_CN.zip

Hi,

Thank you for the excellent library. I am writing a library that harnesses astropy Time classes and I needed TimeInterval and TimeIntervalList classes. Your library is exactly what I need. However, I find that due to floating-point artefacts, the functionality is not always as expected.

My question is, would you consider augmenting the relevant methods with an optional, user-defined tolerance, passed as a parameter?

Details:
There is a problem I'm having with Time classes. Let's say you have a time at t0 and you want to test contains() this for an interval of (t_0, t_1] i.e., interval.contains(t_0). Due to round-off errors and suchlike, the answer may or may not be False.

I assume that a similar error is possible in the native datetime class, or any other problem where the underlying comparison is handled with a float.

I found that I have to do a tolerance check at the edges. As an example, please have a look at my is_in_interval method, which the equivalent of portion's contains():

def is_in_interval(self, time):
        # check upper and lower boundaries for tolerance
        if _are_times_almost_equal(self.start, time):
            # time at starting edge - is edge closed?
            if self._interval.left:
                return True
            else:
                return False

        if _are_times_almost_equal(self.end, time):
            # time at end edge - is edge closed?
            if self._interval.right:
                return True
            else:
                return False

        # Time not edges, do a regular check
        return self._interval.contains(time)

And the _are_times_almost_equal is a simple tolerance check (where EPS_TIME is a very small duration 10 nanosec for me):

def _are_times_almost_equal(t1, t2):
    return abs(t1 - t2) < _EPS_TIME

Edit: Code formatting mishap fixed

Hello and thanks for the great library!

I think many programmers would enjoy this library even more if it had typing information, especially for the exposed api.

Are you planning to add any info with regards to type hints? Would you be willing to merge any PR related to that?

Cheers 🍻

The automatic simplification seems to fail with singletons if it closes exactly one gap. In this case the singleton is only added to the left open interval on the right of the sigleton.
Examples:

P.open(1, 2) | P.open(2, 3) | P.singleton(2)
>>> (1,2) | [2,3)
P.open(1, 3) & ~P.singleton(2) | P.singleton(2)
>>> (1,2) | [2,3)

If the result is stored in a Variable x, further calculations might be faulty:

P.open(1, 3) in x
>>> False

What would be the preferred way to set an optional interval check?

I want to make a callable class, which checks, if the argument is in a given interval. The interval is an optional parameter, such as in these two classes F and G:

import portion as P

class F:
    def __init__(self, interval=None):
        self.interval = interval

    def __call__(self, x):
        if self.interval is None:
            return x*x
        elif x in self.interval:
            return x*x
        else:
            raise ValueError(f'Argument x not in interval. Got {x}')

class G:
    def __init__(self, interval=P.open(-P.inf, P.inf)):
        self.interval = interval

    def __call__(self, x):
        if x in self.interval:
            return x*x
        else:
            raise ValueError(f'Argument x not in interval. Got {x}')

Which implementation (class F or G) is preferable?

Thanks for your help and insights!

>> import intervals as I
>> x = I.open(int(1), int(3))
>> y = I.open(int(4), int(5))
>> x|y
(1,3)|(4,5)

these are open intervals, the bounds are defined as Int instances. So, unlike reals, the two integers abut, there is not an infintity of real points or intervals between them. Surely the result of x|y should be a singleton which is the extend of x.left and y.right not the explicit x|y outcome?

If I coerce this as (1,3) | (3,5) it works. So for the bounds matching it works. It feels like the difference between open and closed intervals has been lost here.

The overlaps() function seems to have some problem with commutativity and atomic intervals.

This example has a problem:

I.closed(1,1).overlaps(I.openclosed(1,2))
False (correct)
I.openclosed(1,2).overlaps(I.closed(1,1))
True (should be False)

But, this example succeeds:

I.closed(0,1).overlaps(I.openclosed(1,2))
False (correct)
I.openclosed(1,2).overlaps(I.closed(0,1))
False (correct)

There is a similar problem with union.

This example has a problem:

I.closed(1,1) | I.openclosed(1,2)
(1,2] (should be [1,2])
I.openclosed(1,2) | I.closed(1,1)
(1,2] (should be [1,2])

But, this example succeeds:

I.closed(0,1) | I.openclosed(1,2)
[0,2] (correct)
I.openclosed(1,2) | I.closed(0,1)
[0,2] (correct)