[DEPRECATION] Discussion on Fused attention and QiGEN about autogptq HOT 5 OPEN

Qbits (intel) PenghuiCheng #660 is another qigen alternative and actively support by Intel.

from autogptq.

1.Anyone still using these?

maybe. No.

2.is fused attention actually faster than marlin even if working properly?

No, I think it's more of a legacy of fused attention and would be good to get rid of.

3.Is SYCL [SYCL] Intel SYCL runtime support for AutoGPTQ #638 a candidate to replace qigen? Intel staff is willing to actively support this in autogptq.

qigen is a kernel that makes inference possible on the CPU. If this SYCL is CPU inferable, it seems like a good idea to remove qigen.

4.With vllm's new marlin kernel that will support almost all groups sizes act-order, do we even need fused attention?
maybe. no

Additionally, it seems like a good idea to remove triton v1 and replace it with triton v2, since all the features of triton v1 are supported by v2 and it is faster.

from autogptq.

Hi @Qubitium ,

We greatly appreciate your interest in QBits. For a comprehensive introduction to QBits, please refer to the RFC. It's worth noting that QBits is still under active development, and we're committed to continuous improvement in both performance and features.

Performance enhancements:

1. Hybrid architecture CPU optimization: We're working on in-depth performance optimization for P/E core scheduling on hybrid architecture CPUs (12th Gen Core processor and beyond).

2. GEMV op optimization: We're also optimizing performance for GEMV-like operations.

3. AVX2 instruction optimization: For client CPUs based on AVX2 instructions, we're continuously optimizing performance.

Feature enhancements:

1. Support for more bit weights: We plan to support more bit weights in the future, such as 2/3 bits, and even 5/6/7 bits.

Regarding pr660 replacing qigen:

wonder if that can pr660 totally replace qigen? if cant, what other efforts should we take?

from autogptq.

I think the current Qbits can replace all parts except 2 and 3 bits of Qigen.
Qigen code:https://github.com/IST-DASLab/QIGen/tree/master

from autogptq.

I think the current Qbits can replace all parts except 2 and 3 bits of Qigen. Qigen code:https://github.com/IST-DASLab/QIGen/tree/master

hi, ITREX will release next version in late may, which support 2/3bit linear

from autogptq.