an invention by xarbulu
Description
I have always been sceptical about AI code assistants such as copilot or gemini. It is obvious that these kind of tools are and will be part of our software engineering life, so let's better learn how to use them the best possible.
Besides that code assistants, I want to play and learn how to use our own IA oriented tools, like the trento MCP server, to see the capabilities and limitations.
Goals
I want to install and start using Gemini code assist to see how this can change my day by day work as a programmer. Try to make questions about our current codebase and see if I can get good results.
I don't expect to spend more than one full day with this, but I think it is worthy!
Resources
https://developers.google.com/gemini-code-assist/docs/overview
Comments
-
1 day ago by xarbulu | Reply
My 1st and 2nd days of this work has been interesting and frustrating at the same level. First of, I have to choose the different AI flavors I want to use. There are some few, so I have chosen one that was easy to install: Gemini. For the code assistant I have played with Gemini Code assistant, and as I have heard from my colleagues lately, the results are bittersweet. In my hand, the interaction I needed to have with the assistant to have a "solid" result, which ended up being a skeleton that I needed to change bottom to top. I did 2 experiments: - Ask the assistant how I could improve a UT test that was testing a piece code with an async call. None of the first answers i was receiving were good enough, so the "conversation" ended up being me telling/teaching the chat different options, so it could continue ellaborating. After a while, I stopped trying, as it started suggesting old solutions back again. None of the given solutions worked out of the box, as a copy/paste, only 1 was a easy plug-in, but the results was not satisfying me.
The 2nd experiment was to create a full end 2 end test group of a new feature I implemented last week. The result was not that different. The initial proposals the chat made had "sense", which means that the logic of the tests was good. It was proposing to test what I would expect. In the other hand, the code structure and the code itself was a mess. After additional time back and forth trying to ask the AI to improve the result, and I ended up using really little from what the AI gave me. I refactored 80% of the given code.
I cannot tell I'm really inspired by all this. I see that it can produce good skeletons, repeat already existing code-patterns and by good writting really predictable code. Which I'm fine with it. I think I will use it more often for certain tasks. Something that I learned is that how to use the AI is as important as the model itself in some cases.
-
1 day ago by xarbulu | Reply
The 2nd experiment and learning has been about using our new Trento AI tool, the MCP server. I configured Trento, MCP server, MCP host and got a Gemini API key. I took my a couple of hours to have everything working. Not that bad. I did different tests, staring with querying read only operations and moving forward with "dangerous" operations (post/put/delete operations) which change the system. These are my conclusions: - There is a huge work ahead of us. The experience is not really pleasant. Whether the model is not good enough or our specification lacks information, the AI was not able to help me in many of my actions. - I needed to have a lot of back and forth to get simple thing as knowing the hosts that didn't have a compliant saptune host. You need to make many questions. But after a while, you get it. Not that bad. - The "dangerous" operations are other story. The is not ready. First comes the confirmation thing. Some operations are asked to confirm about it, others are not. When a confirmation is required, the action is performed regardless. And even in this case, when the action is performed, the AI thinks it wasn't, so it enters a really long loop telling you the action failed because you didn't confirm. - Finally, the product was not smart enough to associate the different host/cluster/sap systems easily. You need to tell it everytime specific resource IDs so it knows what to do. For example, to put the cluster of a certain host in maintenance. If you asked that from a host perspective, the AI would always ask me later to give the exact ID of the cluster, even though the host has this information.
It has been an interesting test. I see the potential and many room for improvements in things that are under our control. We would need to improve our specification and supply more data.
-
about 14 hours ago by xarbulu | Reply
The last day of experiments was more useful. I had 2 challenges. Do some easy refactors on code that I pretty much knew what I wanted and create a new piece of code from scratch (in golang) this time. The refactor part was really successful. More I learn how to write prompts, easier I get good results. In this case, I wanted to replace some functions that were written in and consolidated in other files, importing them. In this case, finally a plug and play result! I simply asked putting many details what I wanted, and the result was perfect! Besides that, all the UT were updated nicely.
The 2nd challenge was interesting as well. I wanted to create a new Trento operator from scratch, a simple one. The first code scheleton included all the main functions I wanted. Good! Many parts of the code were copy/pasted from other parts. After some additional prompts, I was able to remove all the code duplications, and add more details in the code. Finally I have a fully functional operator! And with tests! It is true that this case was really really simple, and I think it wouldn't have taken longer if I did it myself (a lot of copy/paste included). But I'm satisfied with the result!
Similar Projects
This project is one of its kind!