apogee_ntv antononcube: Same. 00:01
SmokeMachine I'm still not very comfortable with that... :( 00:11
apogee_ntv I am but only because I write very long specs and then review the code against the spec after. 00:20
I start with a spec, break it down into tickets each with a plan with the LLM, then assign tickets and review them against the plans, then review unstaged vs the spec and make sure the tests cover it. 00:21
Each plan has a definition of done, the spec has a definition of done.
Original spec is all written by me but can be revised through the exploration (often I find what I write is not ambiguous to me but is ambiguous to the LLM). 00:24
Its not the fastest way to develop with AI but I think it's a way that balances speed of delivery with understanding of what's getting built. 00:25
01:05 MasterDuke joined 01:52 hulk joined 01:53 kylese left
disbot <anagogogo> it's a reasonable way to do things tbh 01:54
<anagogogo> you wouldnt expect much of the AI output to be if you had virtually no more input 01:55
01:57 MasterDuke left 02:15 hulk left, kylese joined 02:46 vasko4535586165 joined 02:48 vasko453558616 left, vasko4535586165 is now known as vasko453558616 03:43 vasko4535586169 joined 03:46 vasko453558616 left 03:48 vasko45355861691 joined 03:49 vasko4535586169 left, vasko45355861691 is now known as vasko4535586169 03:52 vasko45355861691 joined 03:54 vasko4535586169 left, vasko45355861691 is now known as vasko4535586169 04:59 Sgeo left 05:09 charsbar left 05:16 charsbar joined 06:21 dustinm` left 06:33 dustinm` joined 06:39 Guest67 joined 06:48 abraxxa joined 07:09 abraxxa left 07:18 Guest67 left 07:19 Guest67 joined 07:25 leifgunnar joined, Guest67 left 07:42 dakkar joined 07:51 stanrifkin joined 07:54 stanrifkin left
tadzik not using LLMs is immature now? 08:39
09:09 snobber left 09:51 stanrifkin joined 11:56 dakkar left 12:13 dakkar joined 12:45 japhb left 12:50 japhb joined 13:31 vasko4535586169 left, vasko4535586169 joined 13:59 Sgeo joined 14:32 snobber joined 15:07 snobber left 15:26 eseyman left 15:32 snobber joined 15:37 snobber left 15:39 manu_ joined 15:54 stanrifkin left 15:55 dakkar_ joined 15:59 dakkar left 16:04 abraxxa joined 16:14 dakkar_ left
[Coke] finds File::Tudo and thinks he might use that if it supported a markdown file style todo list with checkboxes, with categories (say # headings in the markdown file) 16:51
looks like the current file format is "thing\n--\nthing2\n--\n..." 16:52
17:38 lizmat left 17:50 lizmat_ left 17:51 lizmat joined
disbot <antononcube> Yes, and stupid. I have not forgotten I promised bring concrete examples. 17:55
<antononcube> Got "distracted" from that doing numerics. 17:56
<antononcube> Hm... I have not used tickets yet on personal projects that involve LLMs / AI-agents. 17:57
18:00 jjido joined
[Coke] oh, I was hoping for some nuance or grace in that statement, but just "stupid"? wow. 18:02
There are certainly folks in the community (and here in the chat) who disagree with that apparent stance. 18:03
18:15 jjido left 18:16 iwuvmika left 18:18 iwuvmika joined 18:22 iwuvmika left
lizmat I wonder how stupid it will be when the LLMs will be charging the users for what they really cost 18:23
18:31 hankache left 18:32 hankache joined
japhb Hey y'all ... while I'm busy writing up history and recommendations for our future AI stance, please consciously de-escalate AI discussions. It's important to me that the pot not boil over while I'm cooking. :-) 18:56
apogee_ntv 98% of my LLM usage is local so I pay what they really cost :D 19:16
About $75 on cloud a month then rest local 19:17
I don't really fancy waking up one day to Claude API rates as the only option. The heavily subsidized max plans are nice but I've been closing mine. I had 9 and now I have 1 + $75 openrouter topups. 19:18
So I guess 275 on cloud but I'm closing that last claude sub sometime soon 19:19
19:27 iwuvmika joined
Voldenet tadzik: real serious person wipes their prod databases with claude in 10s, unlike immature people whose systems are stable and boring ;p 19:45
I don't know how to use llms so I plan to dump some prod dbs by hand to pretend I do
s/dump/drop/ 19:46
evalable6 Use of uninitialized value $_ of type Any in string context.
Methods .^name, .raku, .gist, or .say can be used to stringify it to something meaningful.
in block <unit> at /tmp/7W20o3e5lS line 1
apogee_ntv tbh if people give LLMs access to prod DBs this is also pretty stupid :P
Voldenet tbh I pay zero on LLMs, it's my employers who sponsor this, if I had to pay, I'd seriously reconsider whether I should use LLMs 19:47
apogee_ntv I spent far too much on GPUs
Voldenet or yeah, I'd use local models
apogee_ntv I just dont like the subs, they put the usage down multiple times and its like, how am I supposed to judge the value of this product when the product changes every 2 months? 19:48
And I understand why they do it but still, I cant rely on that when my business needs to be able to write code at LLM speeds.
or like when they just deleted fable for a month 19:50
Voldenet funny how people think that writing code is the process that takes the most time 19:52
apogee_ntv Reviewing code takes the most time now
Voldenet which is totally untrue, some patches I make in 5 minutes and then take 10 on testing, building, commiting etc. 19:53
cutting those 5 minutes into 1 minute of generation and 4 minutes of reading doesn't help for a lot of tasks
apogee_ntv Yeah if its a 5 minute task I wouldnt use an LLM
Voldenet and the worst part, some tasks I can do in 5 minutes, but LLMs are not 100% reliable
so, effectively, the task can take even more time 19:54
apogee_ntv Yeah I think balance in all things
If its a smaller task I often just write the code myself, its faster than trying to explain it to a machine.
Voldenet as I'm always saying LLMs are good at some things, bad at others, takes experience to know
apogee_ntv 100% 19:55
Voldenet like don't ask LLM to make an app architecture, they're awful :P
somehow the effect is even sub-junior
apogee_ntv Yeah I always start with my own spec written out, I used to be a lot less specific and I ended up just rewriting things constantly.
Voldenet yeah, now instead of writing code for 5 minutes you write spec for 5 ;) 19:56
apogee_ntv Well yeah I usually write a spec for what would be multiple days of work then it takes about 4hrs to shape that into solid tickets and about 4hrs to review everything.
But it can write the code while I sleep so I can save days 19:57
Often the review ends up creating a repair spec but it turns 5 days of work into like 10-12 hrs + building in the background 20:00
Voldenet I wouldn't feel comfortable with reviewing something this massive 20:02
apogee_ntv It helps that you're reviewing it against a strict spec, not against a vibe.
Voldenet frankly speaking, anything over 500 lines sounds impossible to review reliably 20:03
and when LLMs code you have to actually review details
apogee_ntv Each ticket in the spec may only be 500 lines, you review sequentially not all at once. 20:04
Voldenet like recently I've seen LLM do regex with (\d)+ instead of (\d+)
yeah, but then your N tickets turn into N days of work anyway
apogee_ntv That's not been my experience. 20:05
Voldenet maybe N/k, but still 20:06
apogee_ntv Yeah N/k is accurate
Voldenet Welp, using llms for everything is like searching every standard lib function in docs or google 20:08
you don't need llm to fix a typo :P
apogee_ntv Yeah like I said I wouldn't use them for everything. :D
I usually wont use them for a <1d task because its genuinely slower.
Maybe to help investigate it but not to code it 20:09
You have to have judgement about these things, like any tool. 20:10
Voldenet yes, I can grasp why people wouldn't use it though and the reasons aren't immature 20:11
until people get used to it and tools improve, things will get worse 20:12
companies will reduce workforce giving more tasks to one person, which people like to call brain fry, but I like to call being overworked 20:14
apogee_ntv Yeah I don't think it's immature not to. I think it's immature to attack other people for using it but I'm not going to judge anyone for what tools they use or don't.
As long as they own the code and don't forget where the buck stops. 20:15
I don't like the unrealistic expectations companies are starting to get about how fast AI makes delivery, it's a bit faster from what I can tell, it's not "oh we can just build this in 5 minutes" if you actually plan to use it as more than a throw away. 20:17
Voldenet actually I've heard "now we have llm, so we can build it in 5 minutes" used against the task that would take around 1 or 2 days… even with llm :D 20:19
apogee_ntv Same
Voldenet some people start to get a bit deranged about this 20:20
apogee_ntv You hear it all the time, "why does this still take days? Aren't you using agents?"
Yes do you want my agents to review it and you sign it off or do you want to be sure it will work and not have memory bugs? :D
Voldenet I heard "why does it take over 1 day if it's just one additional `if` of code" before it was LLM-driven ;> 20:25
apogee_ntv lol, because 7h 50m investigating the issue and 10m writing and testing the if? :D 20:26
Voldenet nah, the task actually was to migrate the single-field sql data into complex datatype
that adds additional bool property
and rewrite plenty of code by hand 20:27
apogee_ntv Ah yes
I remember a lot of 1-2 day tasks that genuinely took that long and made 3/-19 LOC.
And getting questioned on why it took that long lol 20:28
Voldenet either way, my point is that managers always expect things to get delivered faster than they can be
apogee_ntv Because it had a huge investigative component obv
Yes
But AI has made *that* worse.
Voldenet bah "i'm out of credits" 20:29
apogee_ntv Same as 'why do reviews take so long now?'
*looks at the PR with 13,000 lines* 20:30
I wonder
Voldenet :)
have you tried splitting the patch into two windows
apogee_ntv OK 8,500 of them are redundant tests.
Voldenet looking at one window with one eye and at the other with second
apogee_ntv I saw one a few weeks ago on a web project it had the test 'returns a 404'... OK 20:31
'does not return a 200', 'does not return a 201',...
Clearly someone told the LLM to make the tests exhaustive :D 20:32
Voldenet I hope the test was also testing the other 200 error codes
just to be sure
apogee_ntv Yeah I was tempted to put a comment saying it forgot to test that it doesnt send a teapot status 20:33
Voldenet is 587 valid error code? Well obviously it isn't, so it should be tested
apogee_ntv Another one is the code duplication 20:36
20:40 abraxxa left
Voldenet you can get agents to find code duplications pretty well 20:40
but yeah, you have to ask for it
apogee_ntv Yeah, they're pretty good at it if you ask, but the 'if you ask' matters. 20:43
Voldenet the "if you ask" is probably the worst part about AI-generated code, normal programmers don't have to think about avoiding duplication
it's semi-autimatic 20:44
apogee_ntv Yeah 100%, there are a lot of things you have to be explicit about. It's the same with picking up a ticket, a normal programmer will just assume it should follow the standards of the repo and reuse whats available lol.
21:17 iwuvmika left 23:40 human-blip left 23:42 human-blip joined

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!