Gemma Lara Savill
Four levels of expertise to harness the power of AI as a Software Enginee
By Gemma Lara Savill
Published at March 10, 2024
AI has been everywhere lately. I have heard a lot about how all the business leaders are visualizing how to use this new tech in their companies. But what about Software Engineers?, what can I do right now to adapt to this new era?
They say good tech should be indistinguishable from magic, but I like to know how the tricks work. I like witnessing magic, but I like making magic even more.
I have some experience with AI, from my time as a mentor of the Self Driving Car Nanodegree at Udacity. Becoming a mentor for this course meant, amongst other things, that you have to first pass the projects of the course.
What I did back then (Jan 2017) looks like ancient history compared to what we have now.
I remember hours and hours of driving a car simulator around a track to gather training data. Then normalizing and preparing the data. After you set up your model, you then had to spend a lot of computing to train it. A LOT. And you had to repeat this many times if the result wasn't good enough, and at first, of course, it wasn't.
I found the result really awesome, as the car was able to self-drive around a track that it had never encountered before, without going off the road. Here is the video of the car caculating the steering angle from the landscape of new track.
GitHub Project 3. Behavioral cloning
But now we have a new AI world. So of course this has got me thinking... what could I do with these new LLMs? I don't need to provide the computing power any more, that has all gone to the cloud. They are pretrained! Out of the box!
It is early days, and we all know this is just the beginning, but so far, as a Software Engineer I can see 4 levels of expertise to investigate and learn:
1. User level. Becoming an expert user:
I would recommend to use it a lot. Learn to use it, experiment.
Learn how to:
- get better results if you give it more context
- create good queries, and how not to ask things
- ask it things without sharing, don't leak sensitive data or company code. Keep things generic.
- reset it when you see it is getting stuck and just start over.
I would suggest you get used to using prompts. This tech is here to stay, and it is going to be everywhere in some form or another.
I think using prompts can help you become a better developer. It is a new tool. The main problem being that there are so many prompts out there. But you try this one and that one, and you start to get the feel of which one is more useful for certain tasks. Benchmark data is important, but I also like to have my own subjective opinion.
I am now having fun with prompts. Right now, I find using a prompt like a strange mixture of searching the internet while debating with myself.
Have you tried any prompt that generates images? It can get weird... as much as it is awesome.
2. Creating a prompt level.
At this level you can learn how to use cloud services to create customized prompts. Here is a free course that walks you through how to use a cloud service: Generative AI Studio provided by Vertex AI on Google Cloud. Using it you can create prompts using Gemini multimodal.
Google Cloud Introduction to Generative AI Studio Course
There are many flavours, tools and brands out there, and more popping up every day. I plan on just getting a handle on the basics: the process, the options, the outcome. I like being aware of the formats. At the moment we have text, images, audio, and video. Possibilities for applications are sort of limitless. For me, this level is just a question of being prepared for when your client or employer, or yourself, want you to create a prompt.
3. Integrating LLMs into your own programs level.
I visualize this level as becoming the prompt, creating the interface between the user and the AI. Being prepared to integrate the API or library into web apps or native apps (Android / iOS / cross-platform). Here you can tune the look and feel of the exchange with the end user. I don't think companies will all want to have boring, similar looking text prompts, they will surely want their own customized interface, specialized to their branding and business needs. This is going to be big. In my opinion, a very interesting area to be involved in. At this level, you can be building the future. Bridging the old world apps into the new AI era.
I have been wanting to build an Android app integration, but to be honest there is such a great example by Philip Lackner, you can just sit and watch. I recommend you have a look, it is only 8 minutes long, and it gives you a feel of what we might be doing any day now, integrating a call to an AI into our programs. Here he creates an app that integrates Google's Gemini Pro into an Android app, to get furniture suggestions for a room, from a photo uploaded by a user. This is a typical API integration: getting your API key and managing the data that is sent and received by the API.
I've Built an Android App With Google's New AI Gemini by Philip Lackner
Sooner or later, I am pretty sure I will be doing this as part of my Android Developer job. And, Gemini Nano will be included into the phone hardware, so the model will be on the device. This will be exciting to test as well. I am looking forward to this.
4. Learn how to train and maintain LLMs level.
This is the level to learn if you really want to be involved in creating this new tech. This is a higher level than just adapting.
For me, this crosses over into the Data Engineer realm. I see the name LLM Ops used. I am not quite sure of the naming, but there is a whole new expertise level here. And it is changing fast.
At this level, you can take a generalist LLM and train it. A bit like what I did in my old course, but you already have a model and the computing power. No more burnt chips or busted pixels on screens (yes, this happened to me, but that is another story).
Here is a free course by Udacity and Comet where I am learning how to build, deploy and maintain LLMs. This involves the whole LLM Ops lifecycle and knowledge of Python.
Udacity LLMOps: Building Real-World Applications With Large Language Models
Understanding how it works under the hood is important to me. I am not really sure that I will be professionally involved at this level, but I certainly want to have enough knowledge to know what it is like to work with LLMs at this level.
I can't help mention this other free course, it really helped me to bridge the gap from my 2017 AI car driving project to the GPT era: Transformer Models and BERT Model with Google Cloud. The magic (science!) that is changing our time.
So this is my experience on adapting to the new landscape right now. As it is shifting all the time, I am concentrating on keeping to the basics and trying not to focus so much on the brands, but on the underlying science. I hope you can find some of my path useful for your own discovery journey.
