[suggestion] Add a client/server mode

[ferdiu]

That would be very helpful to have the possibility to use this software in a client-server manner to let the user be able to use a remote instance where the GPU with running the model will be.

[Belluxx]

Thanks for the input! I'm planning to add support for a llama.cpp remote/local endpoint. Tell me if that should do it.

[ferdiu]

Absolutely yes! That would be of great help! 💪

[Belluxx]

@ferdiu Unfortunately i just checked and the llama.cpp server ignores the echo: true param so I have to wait for this PR to be merged: ggml-org/llama.cpp#15189.

I could generate the response token by token but it would be extremely inefficient and slow.

[ferdiu]

I see, it was opened a while ago. Let's just hope it will be merged soon then.