ValueError: Invalid expression '[ True]', must be integers about ring-attention-pytorch HOT 7 CLOSED

Hey Phil, you're totally good, I was just wondering if I had missed a bug somewhere and thought I'd share my findings here.

I love that you're finding it helpful!

from ring-attention-pytorch.

@fferflo dinner's on me if you ever come to SF! 🤣 i can invite Alex too (he's in south san fran)

from ring-attention-pytorch.

That would be really awesome, although it's quite the trip from Germany so I'm not sure when I'll be able to cash in on the dinner 😄

from ring-attention-pytorch.

@kyegomez ah yea, maybe einx is still a bit new

reverted back to einops for now

from ring-attention-pytorch.

I might overlook something, but it's possible the error stems from passing heads=True rather than an int to RingAttention in https://github.com/kyegomez/Gemini/blob/main/gemini_torch/long_gemini.py#L65 and might appear with einops also.

from ring-attention-pytorch.

@fferflo oh hey Florian! yeah this is my bad, should have checked instead of assuming

let me revert back to einx, been using it in a couple projects now and so far so good!

from ring-attention-pytorch.

@lucidrains @fferflo no invite 😔

from ring-attention-pytorch.