As Large Language Models have risen to the everyday usage, Metal Toad took a few days to have a hackathon and see how much they could accomplish in 2 days and how little code they could actually write. Below are some of the things I've learned.
For common tasks it is works. Mostly.
When doing common tasks that have been done a lot like coming up with CSS or a general http call prompt engineering is 90%+ accurate. The inverse is also true for newer technology or more specific knowledge areas its accuracy was almost zero. One example we ran into was writting AppSync Resolvers in JavaScript. This was specific enough and new enough to be very problematic.
Templates were great
When needing a new component for our messaging app. Being able to ask for a react Component with a set of functions and inputs was great. Once that was complete we would need to update some variable or field names but it was probably 90%+ accurate. I've noticed similar results when asking for python functions. On thing to note is that the smaller the increment and more specific you could be the better the results.
Troubleshooting
Pair programming is an amazing tool for troubleshooting an application. Unfortunately there may not always be someone available to pair program with. In that case using Bard I was able to ask for help reviewing error messages. Asking for alternate code examples to complete a task.
Engineers aren't going away yet
There has been a lot of concern about people being replaced by AI and its something we should all watch out for, but I can comfortably say that unless there is another major leap forward in LLMs, Engineers aren't getting replaced. They are a great tool to improve quality and speed but it can't replace knowing a technology.
Hackathons are a great way for Metal Toad engineers to work with new technologies and build our skills. Using prompt engineering to write as little code as possible was a fun experience. I look forward to learning more during the next hackathon.