vllm on Linux

First, vllm requires the ctypes library.

$ sudo apt install libffi-devel

Use vllm when in production and when you want possibly the highest performance out of your LLMs.

#ai #ai/realworld