Electriclizard solution

[electriclizard]

Hi 🤚!
Here is my simple solution for nlp models inference server.
It works with cpu and gpu. And i've tested it with wrk benchmark tool.
Documentation will be available at /documentation/swagger-ui endpoint.
It also has some future works, like models weights update process.
I also think that inference models with Nvidia triton inference server will be more efficient, but not sure about the deployment with helm chart
I've also tried threading for parallel models call, but still have save rps on benchmarks, it was expecting, but i tried, now working on increasing rps

[rsolovev]

Hey @electriclizard, thank you for the great solution. Here are our test results on grafana dashboard

If you would like to work on your solution further, you can continue optimizing/improving it and re-request our review once done. Any contribution during the challenge period will be taken into account while choosing a winner. Many thanks!