Navigating AI's Environmental Impact

I use large language model completions many, many times every day to help write code and as part of the solutions I build. So, I am in. But I am also unimpressed with the hype, distrustful of the big players monetary incentives, and trying to engage with the ethical and environmental challenges we are creating for ourselves with this technology. In other words, I’m trying to do my job professionally.

This study by Luccioni, Jernite, and Strubell is the first I’ve read that in one place compares and quantifies the environmental impact of types, sizes, and structures of models as well as the relative costs of training to inference. By necessity, it’s using models available on huggingface not the big mega models but the data is certainly directional. https://lnkd.in/em_VS8DU

Some thoughts:

If an existing model is purpose trained for the specific task you are trying to do, it will be much more efficient than a generalized model.

Consider fine tuning pre-trained models within a consortium or open community so the reduction in energy usage on inferences might scale to make up for the additional training costs.

Use smaller models where possible and investigate the model’s structure as that can make a large difference in energy costs to run them.

Limit use of generative models to the specific tasks where they add clear value. Which also means don’t use multi-agent, multi-prompt solutions where more effort on prompt design would allow for a single shot solution.

Engineer code solutions or use alternative machine learning approaches where they work.

Prefer using generative models in design time to help engineer solutions that reduce their use at runtime.