When running tests in a computer under load the following test fails intermittently.

This problem was exposed when I ran the tests using detox. I've seen failures on both the static and exponential timeouts.

"and popular third-party libraries like toolz, fancy, and more-itertools."
but the link is to "funcy" library.

Hi there, neat library!

Can you provide some documentation about how to run the tests?

I installed tox and ran it, and seem to have encountered some setuptools-related errors, and am wondering if there are some other steps I should take. I'm on Mac OS if that's helpful.

Hi! I really like the job you are doing on funcy (even if it lacks a little on the inline documentation side :) ).
It works fine on python 2.7, but on python 3.3 it gives me the following error:

Installing collected packages: funcy
Running setup.py install for funcy
File "/usr/local/lib/python3.3/dist-packages/funcy/colls.py", line 88
return walk(lambda (k, v): (f(k), v), coll)
^
SyntaxError: invalid syntax

This is because in python 3 the lambda function cannot unpack tuple.
I installed it with:
pip install -U funcy

Is there a recommended way to clear funcy's memoize cache for unit tests without needing to use child processes?

I'm not sure whether this function already exists. I'd love to see something like

update([{'a': 1}, {'b': 2}], letter=True)
  # => [{'a': 1, 'letter': True}, {'b': 2, 'letter': True}]

Happy to submit a PR if this doesn't already exist.

So I was wondering is it possible to work with get_in to get a possibility of getting a list of items.

Let me give an example:
Let's say we have the data structure of shopping malls. And we would like to know all addresses of shops, across all malls.

data = {
  "shopping_malls": [
    {
      "name": "gallery1",
      "buildings": [
        {
          "name": "A",
          "shops": [
            {
              "name": "Starbucks",
              "address": "Bejing, gallery1, Building A"
            },
            {
              "name": "Costa Coffe",
              "address": "Whatever"
            }
          ]
        },
        {
          "name": "B",
          "shops": [
            {
              "name": "Starbucks",
              "address": "Bejing, gallery1, Building B"
            },
            {
              "name": "Costa Coffe",
              "address": "Whatever"
            }
          ]
        }
      ]
    },
    {
      "name": "deira_city2",
      "buildings": [
        {
          "name": "A",
          "shops": [
            {
              "name": "Starbucks",
              "address": "Dubai, deira_city2, Building A"
            },
            {
              "name": "Costa Coffe",
              "address": "Whatever"
            }
          ]
        },
        {
          "name": "B",
          "shops": [
            {
              "name": "Starbucks",
              "address": "Dubai, deira_city_2, Building B"
            },
            {
              "name": "Costa Coffe",
              "address": "Whatever"
            }
          ]
        }
      ]
    }
  ],
  "_metadata": {
    "author": "Everyone"
  }
}

So what I would love to have is ability to:

>>> get_in(data, ['shopping_malls', '*', 'buildings', '*', 'shops', '*', 'address'])
["Bejing, gallery1, Building A",  "Whatever", "Bejing, gallery1, Building B",  "Whatever", "Dubai, deira_city2, Building A", "Whatever", "Dubai, deira_city_2, Building B", "Whatever"]

so for now for me, there is no way to work with more complex data structures if they have any list, besides specifying an exact path, have an ability to pull something from nested dictionaries in lists.

I'm not sure if the example above makes sense, so if there is any need for clarification I would love to explain it more.

I understand that get_in wasn't designed in that way, but maybe it's a good idea to think about some capabilities like that in get_in or new function that could handle cases like that.

In the two files, tests/test_funcs.py and README.rst the function double appears but is not defined.

I'm looking for functions called keys and values but cannot find anything in the docs.

First of all I have to thank you for this wonderful library! I have some problems with currying though.

Consider the following function:

def adder(a, b):
   return a + b

Now lets say I want to curry this function to make the following possible

adder = curry(adder)

adder(4)(3)  # 7
adder(4, 3)  # 7

The latter does not work since curried functions in Funcy only accept single argument.

I think a more robust way to implement currying is here: http://mtomassoli.wordpress.com/2012/03/18/currying-in-python/

Edit: found an even better way of implementing this: http://stackoverflow.com/questions/9458271/currying-decorator-in-python

I propose to add decorator which will unpack argument:

    def process(foo, bar):
        ...
    
    map(unpack(process), product(['foo1', 'foo2'], ['bar1', 'bar2']))

This decorator already is in ddt library, but it will be
strange if I will install this module only for this.

Python has atleast 3 great functional programming libraries. These libraries have lots of overlapping functionality:

• https://github.com/pytoolz/toolz
• https://github.com/Suor/funcy
• https://github.com/kachayev/fn.py

Maybe also this:

• https://github.com/Suor/whatever

I would see as the benefit of the whole community if the authors of all these libraries could work more together on a common codebase (I'm willing to participate also in making this happen).

@Suor, @pytoolz, @mrocklin, @kachayev Thank you for the hard work all of you have put into these wonderful packages! What do you guys think about this? Could we make more collaboration between the projects?

Pipe and thread are constructs I use frequently to organize my transformations. I'm wondering if there's a reason you don't appear to have them. They differ with compose in that 1) the order of transformations is reversed, and 2) the transformations are invoked by the call.

Any reason why it doesn't?
I think it should, at least in the form with 2 or 3 arguments.

In [28]: from funcy import iffy
In [29]: foo = iffy({'x',  'y'}, len)
In [30]: foo('x')

The above feels like it should be a valid use of funcy.
Thanks!

The decorator is not very useful for me because I can't see with which arguments called the function.

from typing import NamedTuple

from funcy import print_calls


class Parameters(NamedTuple):
    main_value: int
    second_value: str


@print_calls
def process(parameters: Parameters):
    pass


process(Parameters(1, '10'))
process(Parameters(2, '20'))
process(Parameters(3, '30'))

Result:

Call process(Parameters(main_value=...)
-> None from process(Parameters(main_value=...)
Call process(Parameters(main_value=...)
-> None from process(Parameters(main_value=...)
Call process(Parameters(main_value=...)
-> None from process(Parameters(main_value=...)

It would be great if I can define how long should be argument representation

Maybe it's to specific, but what do you think of a function that acts like an inverse of zip, to split a list of tuples into multiple lists
izip(list) = zip(*list)

My use case was retrieving multiple values from a dict/json

names, sizes = izip[(person['name'], person['size']) for person in people]

I'm trying to mix positional and keyword arguments:

import funcy


@funcy.cache(10)
def func(a, b, c):
    return a + b + c


func(1, b=2, c=3)

And I am getting next exception:

Traceback (most recent call last):
  File "/home/adr/projects/rogers-qa-analytics/tmp/tmp.py", line 9, in <module>
    func(1, b=2, c=3)
  File "/home/adr/projects/rogers-qa-analytics/.env/lib/python3.6/site-packages/funcy/calc.py", line 121, in wrapper
    result = func(*key, **kwargs)
TypeError: func() got multiple values for argument 'b'

It looks like the problem is here:

def cache(timeout, key_func=None):
    """Caches a function results for timeout seconds."""
    if isinstance(timeout, int):
        timeout = timedelta(seconds=timeout)

    if key_func is None:
        key_func = lambda *a, **kw: a + tuple(sorted(kw.items())) if kw else a

    def decorator(func):
        cache = {}

        @wraps(func)
        def wrapper(*args, **kwargs):
            key = key_func(*args, **kwargs)
            if key in cache:
                result, timestamp = cache[key]
                if datetime.now() - timestamp < timeout:
                    return result
                else:
                    del cache[key]

            result = func(*key, **kwargs)  # <-- here should be *args instead of *key
            cache[key] = result, datetime.now()
            return result

        def invalidate(*args, **kwargs):
            cache.pop(key_func(*args, **kwargs))
        wrapper.invalidate = invalidate

        def invalidate_all():
            cache.clear()
        wrapper.invalidate_all = invalidate_all

        return wrapper
    return decorator

I'm not sure about the name but the idea is to return the leaf node for given key in mapping of (possible deeply nested) mappings.

from collections import Mapping

def leaf(obj, key):
    item = obj.get(key, None)

    if item is None or isinstance(item, Mapping):
        for subkey in iter(obj):
            item = obj.get(subkey, None)
            if isinstance(item, Mapping):
                try:
                    return leaf(item, key)
                except KeyError:
                    pass
    else:
        return item
    raise KeyError(key)

Usage:

d = {
    1: 'first',
    2: {
        3: 'second',
        4: 'third'
    }
}

leaf(1)  # 'first'
leaf(2)  # KeyError
leaf(3)  # 'second'

Two P's. :) Perhaps an alias could be made for the misspelt version, for a deprecation period?

users_over_21, users_under_21 = split(lambda user: user.age >= 21, users)

thoughts? doing it manually is cumbersome, but maybe you have a better way.

over_21 = lambda user: user.age >= 21
users_over_21 = filter(over_21, users)
users_under_21 = filter(complement(over_21), users)

i'd be happy to implement.

We ran into an issue where we'd still like an exception to be memoized and return the exception when using the memoize decorator. A way to specify an error handler function would be appreciated. Or provide a list of exception types which should be cached.

first([None]) equals to first([]).

Related thread on SO.
Turns out like this is not trivial issue.

Hello,

Is there something in this library to do this:

'my name is vignesh'.replace('vignesh', 'matt').replace('my', 'your')

I've written for loops, regex and reduces to implement this, but its very repetitive to rewrite each time. What do you recommend?

Issues with for loops and reduces include:

• Reduces are not recommended by Guido.
• The ordering matters, since they are not replaced at once.

Regex works, but its not very functional. Although I guess it doesn't matter if the implementation itself is function, just whether the API is in this case, right?

First of — I really like some of the abstractions the library introduces. But exploring the library in python/ipython REPL shell is really hard without proper docstrings.
compare

In [5]: funcy.repeat?
Type:       type
String Form:<type 'itertools.repeat'>
File:       /Users/teferi/git/eksmo/env/lib/python2.7/lib-dynload/itertools.so
Docstring:
repeat(object [,times]) -> create an iterator which returns the object
for the specified number of times.  If not specified, returns the object
endlessly.

to

In [6]: funcy.retry?
Type:       function
String Form:<function retry at 0x1063a1848>
File:       /Users/teferi/git/eksmo/env/lib/python2.7/site-packages/funcy/flow.py
Definition: funcy.retry(*dargs, **dkwargs)
Docstring:  <no docstring> 

Reading the latter really doesn't shed any light on how to use it. I understand that it's a lot of work — to properly document the whole thing, but it would be really nice to have such descriptions.

Also I'm willing to offer some help with that some time later (if that's ok =))

Using the @once decorators on object methods instead of functions does not work (at least on 1.5, appologies if this was fixed in 1.6, I can't upgrade right now). I'm getting TypeErrors on the number of arguments passed to the wrapped method. Is this supposed to work? If not, I'd try and write a patch.

Logically obvious limitation without an obviously documented solution.

Suppose a function decorated by a decorator created with @funcy.decorator actually has arguments named _args, _kwargs, or _func (it shouldn't, but supposing it did).

Reasonably and correctly, call._args, call._kwargs, and call._func still resolve as expected and documented - good so far.

An intuitive solution if you did want to inspect those arguments would be to call @decorator's __getattr__(self, ...) manually.

But that unconditionally sets self.__dict__[...], so for example doing call.__getattr__('_args') can break call(), because call._args gets replaced.

So the only safe way to access those arguments is to manually use the undocumented funcy.decorator.arggetter directly, or to manually implement the same logic of looking up the argument from call._args and call._kwargs based on the signature gotten from call._func.

Actually, I think this problem might possibly effect all of the dunder methods too.

Anyway, I think this could be solved by either:

1. Just officially saying that as a special case, accessing those arguments to the wrapped function using the same built-in logic for getting arguments is completely unsupported.

2. Officially documenting funcy.decorator.arggetter as the way to get those arguments.

3. Using self.__dict__.setdefault instead of self.__dict__[...] in __getattr__, and documenting manually calling call.__getattr__ as the way.

4. Exposing some sort of new method on the Call class like call._getarg, which could use either the 2nd or 3rd option internally.

Thoughts?

See:

$ pip install funcy
Downloading/unpacking funcy
  Downloading funcy-0.5.5.tar.gz
  Running setup.py egg_info for package funcy
    Traceback (most recent call last):
      File "<string>", line 16, in <module>
      File "/Users/xxx/.virtualenvs/ccav/build/funcy/setup.py", line 5, in <module>
        version=open('VERSION').read().strip(),
    IOError: [Errno 2] No such file or directory: 'VERSION'
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):

  File "<string>", line 16, in <module>

  File "/Users/xxx/.virtualenvs/ccav/build/funcy/setup.py", line 5, in <module>

    version=open('VERSION').read().strip(),

IOError: [Errno 2] No such file or directory: 'VERSION'

----------------------------------------
Cleaning up...
Command python setup.py egg_info failed with error code 1 in /Users/xxx/.virtualenvs/ccav/build/funcy
Storing complete log in /Users/xxx/.pip/pip.log

cached_property does not support overriding since it directly writes into the target instance's dict, as in the following example:

import funcy


class A:
    @funcy.cached_property
    def p(self):
        return 1


class B(A):
    @funcy.cached_property
    def p(self):
        return 2

    @property
    def p2(self):
        return super().p


b = B()
b.p2
>> 1
b.p
>> 1
b.__dict__
>> {'p': 1}

This issue should at least be documented if not fixed.

In [6]: d = {'a': {'b': 'carrot'}}
In [7]: funcy.get_in(d, ['a', 'b', 'c'])
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-7-290dd7587deb> in <module>()
----> 1 funcy.get_in(d, ['a', 'b', 'c'])
/Users/matthewwillian/zentreefish/.inenv/zentreefish/lib/python2.7/site-packages/funcy/colls.pyc in get_in(coll, path, default)
    226     for key in path:
    227         if key in coll:
--> 228             coll = coll[key]
    229         else:
    230             return default
TypeError: string indices must be integers, not str

I think the correct behavior here should be to return None (or whatever default is set to). The obvious way to fix this would be to return default if coll is a string. Is that the preferred solution?

I think making all functions from funcy automatically curried would simplify a vast number of use cases. For example, for an extract_int example (where extract_int("Hello, 123! We are 456!") => [123, 456]), without autocurry, the current best way to do this is:

from funcy import map, partial, rcompose, re_all
extract_int = rcompose(partial(re_all, r'\d+'), partial(map, int))

Of course, we could use autocurry on them ourselves - but this doesn't seem to work well with some functions (I tested only with map and re_all, surprisingly re_all works):

>>> from funcy import map, autocurry
>>> map = autocurry(map)
>>> map(int)([1, 2, 3])
# .... blah error stuff ....
TypeError: map() requires at least two args

This and other such problems could be averted by having all functions (at least those in the library) automatically curried.

I love where. What about another similar function, that takes predicates instead of values? Not sure what to call it, but it behaves like this:

def filterwhere(mappings, **preds):
    """ Like `where` but instead of value equality checks, use predicate functions.

    .. doctest ::

        >>> mappings = [{'foo': 'a', 'bar': 16},
        ...  {'foo': 'b', 'bar': 32},
        ...  {'foo': 'd', 'bar': 128, '': ''},
        ...  {'foo': 'e', 'bar': 123, 'baz': 'quux'},
        ...  {'foo': 'c', 'bar': 6400},
        ...  {'foo': 'f', 'bar': float("inf")},]
        >>> x = filterwhere(mappings, bar=lambda bar: bar < 100)
        >>> assert x == [{'foo': 'a', 'bar': 16}, {'foo': 'b', 'bar': 32}]
        >>> y = filterwhere(
        ...     mappings, bar=lambda n: n % 2 != 0 and n < 1000)
        >>> assert y == [{'foo': 'e', 'bar': 123, 'baz': 'quux'}]

    :param mappings: A sequence of mappings, for example a list of dicts.
    :param preds: Any number of predicate functions. Each mapping must
        pass `pred(mapping[key_of_mapping])` for each keyword argument pair of
        `key_of_mapping=pred`.
    :return: A sequence of mappings.
    """
    match = lambda m: all(pred(m[k]) for k, pred in preds.items())
    return filter(match, mappings)

It will produce a better error message in Python 3.

Raise from

try:
    0/0
except Exception as e:
    raise ValueError from e

Result

Traceback (most recent call last):
  File "<input>", line 2, in <module>
ZeroDivisionError: division by zero
The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<input>", line 4, in <module>
ValueError

Just raise:

try:
    0/0
except Exception as e:
    raise ValueError

Result:

Traceback (most recent call last):
  File "<input>", line 2, in <module>
ZeroDivisionError: division by zero
During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<input>", line 4, in <module>
ValueError

Hi all,

I propose to add the raise_last_exception argument to fallback function:

def fallback(*approaches, raise_last_exception=False):
    """Tries several approaches until one works.
       Each approach has a form of (callable, expected_errors)."""
    for approach, next_approach in funcy.with_next(approaches):
        func, catch = (approach, Exception) if callable(approach) else approach
        try:
            return func()
        except catch:
            if not next_approach and raise_last_exception:
                raise 

To make those transducers even less meaningful ;)

>>> foo = iter(range(5))
>>> list(takewhile(lambda x: x <=2, foo))
[0, 1, 2]

By this point, one would expect that the foo iterable would have [3, 4] left.
Instead, we get:

>>> list(foo)
[4]

This is because 3 is burned (consumed) by the takewhile predicate, and in case of False condition it is never returned to the original iterable.

http://funcy.readthedocs.org/en/latest/seqs.html#group_by

says it returns a dict, when it actually returns a defaultdict(list)

I know there's a Python 3.3 Issue already (Issue #1) but I'd like to see support for Python 2.6 as well. I don't think it's unreasonable to support Python 2.6, 2.7, 3.3 and PyPy? I've tested funcy against Python 2.7 and PyPY 2.2 successfully so far.

I propose that something like Lodash's _.uniq could be useful, especially with an optional "key" function similar to Lodash's _.uniqBy.

Simply using list(set(collection)) doesn't preserve the original order.

This code:

mappings = [{ 'a': 4 }, { 'b': 2 }]
print(list(where(mappings, b=2)))

Raises this exception currently:

KeyError: 'b'

Is this intended behavior? It feels like, logically, it should just skip over items that don't have necessary keys in them as they, by definition, don't match the conditions.

Hi,

every time I try to code in a more functional manner in Python I find myself writing this little function sooner or later:

flip = lambda f: lambda x,y: f(y,x)

it's especially useful with currying, which is already provided by the library, so I think it would be a good addition. A very simple example:

has_suffix = curry(flip(str.endswith))
filter(has_suffix("a"), ["ab", "ba", "cd"])

One thing to think about is the semantics of flip when used on a function with more than two arguments and on functions with **kwargs, especially the former case is interesting, as it could mean:

flip = lambda f: lambda a,b,*args: f(b,a,*args)

or

flip = lambda f: lambda *args: f(*(args[1:] + [args[0]]))

The latter meaning would be consistent with for example rotatef from CL, but on the other hand it could be less practical.

In my case, PyCharm has trouble understanding things coming from funcy. Specifically, it fails to understand importing directly from the funcy package rather than the specific sub-modules, i.e. from funcy import merge causes it to mark the import as problematic. I imagine it has other issues too, I haven't investigated further (yet).

I can think of several ways to improve the situation:

1. Re-work the fancy importing mechanisms so that static analyzers do a better job of understanding them.
2. Include stubs which will directly inform static analyzers about the structure of the funcy library and its contents
3. Contribute stubs for funcy to the typeshed project, which is used by various static analyzers and IDEs.

I'd be happy to help with this, if the maintainer(s) are open to it.

Hi,

The following code works fine before 1.11.0

@funcy.retry(5, lambda attempt: attempt * 2)

Now it's broken with the following exception

File "/Users/ykoasanto/Veracode/Agora/Laputa/agora-ecsinfra/ecsinfra/service/deploy_helpers.py", line 196, in poll_ecs_for_steady_state kankyo_name, service_name) File "/Users/ykoasanto/Veracode/venv/ecsinfra/lib/python2.7/site-packages/funcy/decorators.py", line 38, in wrapper return deco(call, *dargs, **dkwargs) File "/Users/ykoasanto/Veracode/venv/ecsinfra/lib/python2.7/site-packages/funcy/flow.py", line 93, in retry errors = _ensure_exceptable(errors) File "/Users/ykoasanto/Veracode/venv/ecsinfra/lib/python2.7/site-packages/funcy/flow.py", line 122, in _ensure_exceptable return errors if is_exception else tuple(errors) TypeError: 'function' object is not iterable

I think the removal of the check for a list type here have caused this issue:

8a13537#diff-64922fc77aa802c10c7245a11b0a213bL95

Do you think you could provide an example in the docs on how to use Easy Decorators?

Context

I have encountered situations when i use funcy.project for entities which implement dict interface and it cause some implicit errors. I am not sure if it actually funcy's bug or bug of dict interface implementation of entities i use it on.

If explanation provided in example paragraph is not enought i can later try give some exact working .py fines which reproduce such behavour (type if need).

Example

djangorestframework ModelSerializer class (or some of it's parents) has initial_data attribute, which is django.http.request.QueryDict

I had following scenario trying to project it:

ipdb> pp self.initial_data
{'archived': 'false',
 'short_title': 'TT',
 'ticket_fare': '1.00',
 'ticket_type': '2',
 'title': 'test title'}

ipdb> type(self.initial_data)
<class 'django.http.request.QueryDict'>

ipdb> pp generic_relation_field_names
['history_entries']

ipdb> type(generic_relation_field_names)
<class 'list'>

ipdb> funcy.project(self.initial_data, generic_relation_field_names)
*** TypeError: expected string or bytes-like object

With following traceback:

File ".../api/serializers/abstract.py" in _get_initial_generic_relation_fields
  104.         return funcy.project(self.initial_data, generic_relation_field_names)

File ".../.venv/lib/python3.6/site-packages/funcy/colls.py" in project
  245.     return _factory(mapping)((k, mapping[k]) for k in keys if k in mapping)

File ".../.venv/lib/python3.6/site-packages/django/http/request.py" in __init__
  407.         for key, value in limited_parse_qsl(query_string, **parse_qsl_kwargs):

File ".../.venv/lib/python3.6/site-packages/django/utils/http.py" in limited_parse_qsl
  410.         pairs = FIELDS_MATCH.split(qs, fields_limit)

i had similar problem with Serializer.data attribute which is <class 'rest_framework.utils.serializer_helpers.ReturnDict'> but i can't say if they absolutely same, because i don't have it's traceback and have not time to reproduce and provide it.

Software:

Python==3.6.5
Django==2.1.1
djangorestframework==3.8.2
funcy==1.11

Actual request

Either:

• fix this behaviour
• add assert that type of passed argument is actually dict (wich i suppose is not a solution since it is probably break back compatibility)

Thanks for the great package!

I am looking for a way to do join_with on binary tuples. Something like

join_with(list, [{1:2}, {1: 3}])
## {1: [2, 3]}

But with tuples:

join_with(list, [(1,2), (1, 3)])
## {1: [2, 3]}

What would be an idiomatic way to achieve this with funcy?

Thanks.

Hey there,

This library is a dependency for PyLDAVis and I'm having this issue running on Python 3.5.1

funcy/decorators.py:56: DeprecationWarning: inspect.getargspec() is deprecated, use inspect.signature() instead

https://github.com/Suor/funcy/blob/master/funcy/decorators.py#L56

It is also being used in other parts of the code in the repo.

Example implementation:

def deep_dict(seq, f=dict):
    def pair_f(pair):
        k, v = pair
        if isinstance(v, (list, tuple)):
            return k, deep_dict(v, f)
        return k, v
    return f(map(pair_f, seq))

Usage:

seq = [
    (1,
        (
            (2, 1),
            (3, 1),
            (4, 1)
        )
    ),
    (5, 1)
]

deep_dict(seq) == {1: {2: 1, 3: 1, 4: 1}, 5: 1}

This needs some tuning though to be able to cope with funcy's schematics.

>>> join_with(concat, [{1: [1,2]}, {1: [2,3]}])
{1: [[1, 2], [2, 3]]}

I was expecting

{1: [1, 2, 2, 3]}

Hi,

when I use @cached_property on a class function it won't get documented by sphinx autodoc.

Is there a way to fix that or a workaround ?

Thanks,
bastian

found following notice in the spinx docs:

If you document decorated functions or methods, keep in mind that autodoc retrieves its docstrings by importing the module and inspecting the doc attribute of the given function or method. That means that if a decorator replaces the decorated function with another, it must copy the original doc to the new function.
From Python 2.5, functools.wraps() can be used to create well-behaved decorating functions.