Add flash-attn to requirements.txt

[lonestriker]

Speedup, especially as context grows.

[turboderp]

I'm not sure what the situation is currently on Windows. I kept it optional for ExLlama because it can be a little tricky to get it up and running, even on Linux where it sometimes takes 15-30 minutes to install if it needs to compile, but especially on Windows. Or at least that used to be the case.

[JohanAR]

I think it should work on Windows too now. Ooba installs wheels from https://github.com/bdashore3/flash-attention for Windows, and from https://github.com/Dao-AILab/flash-attention for Linux. Not so sure about the package on pypi though.

[blakejrobinson]

flash-attn did not install on Windows for me, so worth investigating before integrating if the intent is for this to be nice and lite and user-friendly.

  Using cached flash_attn-2.3.5.tar.gz (2.3 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [17 lines of output]
      Traceback (most recent call last):
        File ".venv\lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 353, in <module>
          main()
        File ".venv\lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File ".venv\lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
        File "~\AppData\Local\Temp\pip-build-env-f3yoy51w\overlay\Lib\site-packages\setuptools\build_meta.py", line 325, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
        File "~\AppData\Local\Temp\pip-build-env-f3yoy51w\overlay\Lib\site-packages\setuptools\build_meta.py", line 295, in _get_build_requires
          self.run_setup()
        File "~\AppData\Local\Temp\pip-build-env-f3yoy51w\overlay\Lib\site-packages\setuptools\build_meta.py", line 480, in run_setup
          super(_BuildMetaLegacyBackend, self).run_setup(setup_script=setup_script)
        File "~\AppData\Local\Temp\pip-build-env-f3yoy51w\overlay\Lib\site-packages\setuptools\build_meta.py", line 311, in run_setup
          exec(code, locals())
        File "<string>", line 8, in <module>
      ModuleNotFoundError: No module named 'packaging'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

[turboderp]

packaging is a fairly common module, but it looks like an overlooked dependency in the flash-attn package. Either way as long as flash attention is optional I'd really rather not make it a requirement.

[blakejrobinson]

I'd agree, nice to have it optional. I do actually have packaging installed, so not really sure. I gave up trying to get flash-attn working as it was just causing massive headaches.