Hi all, very new to the LlaMa deployment scene, was just wondering how i could deploy the model with a dual GPU set up. Would this be automatic as long as DLLAMA_CUBLAS is enabled? Also would anyone happen to know the performance benchmarks of such a setup versus a single gpu setup?
Hi all, very new to the LlaMa deployment scene, was just wondering how i could deploy the model with a dual GPU set up. Would this be automatic as long as DLLAMA_CUBLAS is enabled? Also would anyone happen to know the performance benchmarks of such a setup versus a single gpu setup?