01:33 librasteve_ left 04:27 kjp joined 07:40 melezhik joined 10:29 melezhik left 11:17 melezhik joined 11:35 librasteve_ joined
melezhik [Coke]: ab5tract: restarted o10r, it was down , please restart your agents when you can thanks 11:55
12:24 melezhik_ joined
melezhik_ ||| TESTS STAT: time: 001h | tests total: 2389 | finished tests: 1731 | sent to queue: dist=10 / redist=0 | agents cnt: 1 12:24
12:33 melezhik_ left 12:41 melezhik_ joined 12:45 melezhik_ left 12:46 melezhik_ joined 12:49 melezhik_ left
[Coke] looks like we could use more articles for advent. Anyone have any interest in writing an article (newcomers welcome), or suggestions for topics for others to pick up? 13:48
blin run on the unicode 17 run found one casualty, bisected to the nqp bump. 14:05
"Unicode::Security"
lots of "this kind of whitespace is not allowed in a string: '"\n"'" 14:06
melezhik: restarted 14:18
"this kind of whitespace is not allowed in a string: '"\n"' near 5511" 14:22
that "near" is not helpful.
oh, weird, that's coming from src/core.c/Rakudo/Internals/JSON.rakumod 14:24
ok, it's not his code, it's his data. if you zef look (running on rakudo HEAD), you can duplicate the error with: 14:29
raku -MJSON::Fast -e 'dd from-json "resources/data/confusables.json".IO.slurp'
that file is 18,845 lines long. 14:30
lizmat could it be the error is because something used to be an error, and is no longer? 14:31
[Coke] added github.com/JJ/raku-unicode-security/issues/6 14:35
lizmat: I would imagine it's the other way around?
lizmat afaik, unicode 17 did not invalidate any codepoints ? ShimmerFairy ??
ah, maybe it added some more whitespace ? 14:36
[Coke] But also, not clear if this is a bad json file that we are now only finding out is bad, or if it's something in JSON::Fast, or if it's something in that internals code.
ab5tract melezhik: just saw your message and restarted 14:55
melezhik Thanks 14:58
In the future I will make agents continue to work even if o10r is offline 14:59
ugexe [Coke]: strange, I dont understand why you'd be seeing R:I:JSON in an error when you are using JSON::Fast in that example 15:08
a quick thing to try is using `dd JSON::Fast::from-json ...` instead of `dd from-json` to make sure it isn't trying to use R:I:JSON for some odd reason 15:10
ShimmerFairy lizmat, [Coke]: At first glance I see that Unicode::Security needs to generate data from Unicode, so it's possible that data needs an update. I had collation test failures (including in a non-generated test file) that made me think something major had changed, but discovering the (for some reason) separate collation update script in MoarVM fixed everything. 15:16
ugexe this kind of whitespace is not allowed in a string: '"\n"' near 5511 15:21
in sub unjsonify-string at /Users/nlogan/.rakubrew/versions/moar-blead/install/share/perl6/site/sources/F13CDD097310A0775131666979B65ADF692574DD (JSON::Fast) line 570
so when using JSON::Fast to get that error it does not actually mention R:I:Internals 15:22
15:29 melezhik_ joined
melezhik_ [Coke]: your agent version is a bit old, could you please upgrade it manually ? `docker exec -it agent bash /opt/brownie/agent/upgrade.sh` 15:29
I am going to add auto upgrade soon, meanwhile looks like you build agent from Dockerfile and got old cache data 15:30
thanks 15:31
or, you rebuild from Dockerfile without latest commits from git 15:34
ShimmerFairy I found the issue here: github.com/JJ/raku-unicode-securit....json#L651 15:35
This string literally includes a GCB=Prepend character, which since Unicode 15.1 creates a grapheme that swallows the closing quotation mark, making it continue to the illegal literal newline. 15:36
I think JSON::Fast might need an update so that from-json works on the codepoint level (that is, not use Strs for input text), to avoid this issue. 15:40
m: say "ൎ" 15:41
camelia ===SORRY!=== Error while compiling <tmp>
Unable to parse expression in double quotes; couldn't find final '"' (corresponding starter was at line 1)
at <tmp>:1
------> say "ൎ"<HERE><EOL>
expecting any of:
argument list
…
ShimmerFairy m: say "\x0D4E".raku 15:42
camelia "ൎ"
ShimmerFairy I don't think JSON::Fast uses .raku itself to escape text in its to-json method, (and it wouldn't help with 3rd party JSON anyway), but you can see how a similar problem would arise if you tried to generate Raku string literals. 15:43
So while it's unrelated, this is a good reminder that Str.raku ought to be mindful of "problematic" graphemes at the start and end of text (graphemes which would gobble up the quote mark they're next to). 15:44
15:49 melezhik_ left, melezhik_ joined 15:57 melezhik_ left, melezhik_ joined 16:05 melezhik_ left, melezhik_ joined 16:10 melezhik_ left, melezhik_ joined
ab5tract ugexe: is there a trick for installing zef itself into a custom site directory? I tried r -I. bin/zef -to=$RAKULIB install . where $RAKULIB points to (a subdirectory of) my development installation of rakudo 16:23
But it comes back with All candidates are currently installed
ugexe i guess it isn't ideal in this situation, but it is telling you zef is already installed (at the same version). presumably that is in the default cur site though 16:25
ab5tract ah, ok let me try to force the install and see what comes of that :) 16:26
ugexe you could --force-install which should get around that, but it wouldnt be ideal for other modules that actually have dependencies (since --force-install would also force install dependencies, of which zef has none_
[Coke] ugexe: I did a search for the error string, and don't think I verified the stack trace itself. could be some duplicated code in JSON::Fast.
ab5tract ugexe: congrats on 1.0.0 btw!@ 16:27
ugexe maybe it'll hit 2.0.0 in another 10 years 16:28
lizmat fwiw, the RI::from-json is an only slightly modified version of JSON::Fast
in other words: only a few lines are *not* duplicated
ab5tract Hmm.. now I see: Could not find in: and then $RAKULIB is missing from the search path (clearly, or otherwise it would presumably find what it was looking for) 16:32
lizmat: I'
I'm trying to test Hash::Ordered + that patch I reverted, btw
lizmat and? 16:33
it worked for me with the patch reverted
ab5tract Right now I'm trying to get zef working to install the dependencies of Hash::Ordered 16:35
lizmat zef install Hash::Ordered works for me out of the box with all of its dependencies? 16:36
ab5tract Yeah, but I'm trying to test it against a specific raku installation
maybe it doesn't matter where the interpreter is installed, come to think of it 16:37
ugexe you probably have to do `RAKULIB=foo zef install -to=foo .`
actually that would be `RAKULIB=inst#foo zef install -to=foo .` 16:39
since -to=foo is actually -to=inst#foo
(and you could indeed write -to=inst#foo)
ab5tract hmm, those were the settings I used
RAKULIB is already an exported global variable in this fish session 16:40
so it worked fine to install zef itself, but using that installation doesn't seem to work
lizmat: I don't know whether this is because zef installed Hash::Agnostic (Hash::Ordered's sole dependency) with a different raku or what, but r -I. t/01-basic.rakutest works herew 16:41
with the patch that was reverted put back in place
ugexe RAKULIB=inst#foobar bin/zef --help 16:42
that worked for me
ab5tract presumably export RAKULIB=inst#foobar should be equivalent, right? or did I miss something special about that bash syntax for all these years? 16:43
ugexe i dunno, what does `echo $RAKULIB` show? 16:45
ab5tract #inst/my/installation/site
I'm using fish and there's no equivalent to FOO=what cmd
ugexe in your export maybe you need to quote the string 16:46
ab5tract so I'm just double checking that from the cmd point of view, it's all just environment vars
ugexe also '#inst/my/installation/site' is a confusing example, can you give the actual text?
it isn't clear if it is intended to have a inst# prefix and that got moved
ab5tract #inst/home/user1/code/raku/x.core/raku-prefix/site 16:47
ugexe and your export statement is what?
ab5tract set -gx RAKULIB "inst#/home/longwalker/code/raku/x.core/raku-prefix/site"
ugexe lol yeah i wonder how it ends up with what it does
ab5tract yeah, kind of funky 16:48
luckily I can replicate the issue just using the system zef
probably that won't always be the case, if precompilation changed drastically or something, but it seems to be working now 16:49
lizmat: which is to say, I've reproduced the bug
ugexe you can ensure it is not using the system zef by grepping the output of --help for "CONFIGURATION" 16:50
ONFIGURATION /Users/nlogan/repos/zef/foobar/resources/B3FCDE14C18BABB5151D13E5EC63A15CCF354CB4.json
ab5tract okay, this makes a lot of sense. And we should add a test for this case
ugexe that is what mine shows for example when using a custom repo foobar
with fish i think you can do `env RAKULIB="inst#foobar" zef ...` 16:51
ab5tract the dispatcher is choking on that new istype check when ::?ROLE is used in the dispatch of gist, but probably it's problematic for other methods too
16:58 melezhik_ left, melezhik_ joined
ab5tract ugexe: cheers, thanks for your help! 16:59
17:08 melezhik_ left, melezhik_ joined
librasteve_ dear folk of the core - is anyone up for an advent article, I am getting worried its going to lack the gravity of previous years without some deep and serious shit 17:11
ab5tract librasteve_: I 17:19
will sign up for one (no idea why I fat finger on this particular keyboard so often 😓 17:20
I really appreciate you driving it this year. not an easy task, I imagine! 17:23
melezhik_ [Coke]: ab5tract: patrickb: I've added agent auto upgrade - gist.github.com/melezhik/dd3616d06...2d353a9d68 17:24
please restart your agent from the latest commit and you will no longer to do that to upgrade , it will happen in a future automatically 17:25
ab5tract That’s a bit of a security scare for me, while also sounding quite convenient 17:27
17:30 melezhik_ left, melezhik_ joined 17:36 melezhik_ left, melezhik_ joined
ab5tract melezhik_: After considering for a bit, I don't really want that kind of auto-update on my box for the time being 17:46
&afk
librasteve_ ab5tract: ++ on the advent post - thanks! 17:50
ab5tract no problem :)
I'll make it official tonight or tomorro
[Coke] Agreed, I don't want auto update. 17:56
adds a level of trust required with the server. 17:57
ShimmerFairy: so if I understand right, that JSON should not be working under Unicode 17? 17:59
18:08 melezhik_ left, melezhik_ joined
melezhik Ok. Sorry I thought this is what everyone wanted ? At least patrickb asked me this feature 18:10
No problem I am reverting this
And go back to manual updates
18:14 melezhik_ left, melezhik_ joined
melezhik Reverted github.com/melezhik/brownie/commit...1d22f2eb67 18:19
18:19 melezhik_ left
melezhik ^^ [Coke] ab5tract 18:24
[Coke]: anyways please upgrade to the latest version the agent , as version 0.0.38 lacks some updates I wanted to test , thanks 18:34
patrickb I'm running the agent in a qemu VM I've set up solely for this purpose. That's why I don't have much of a security issue with auto update. But fundamentally I believe it doesn't make much of a difference for the me because at least I don't read the code before I do an update anyways 18:46
[Coke] "but you could" 18:47
patrickb Also I think putting the agent in a VM is a good practice as containers are potentially insecure and an eco toaster is running loads of untrusted code. I wouldn't feel comfortable running that on a machine I do anything else with. 18:48
Just to make that clear: I'm approving of not having auto-update when it gives any user discomfort. 18:50
melezhik I guess I support both point of views. Anyways , as code is not quite stable yet , and I did not want to be seemed as annoying) pinging often people “please do upgrade”, that was the idea with auto upgrade , but if manual updates are ok - let’s leave them ) 18:52
patrickb Is there potential interest in a VM image for this? If there is, I'd be willing to create one. 18:53
melezhik Also wise to run containers in rootless mode if someone wants even tighter security )
Yep, think it’s possible to create a qemu image based on Dockerfile 18:54
Which is not the case for containers across restarts 18:56
patrickb huh? The VM should run on bare metal. Where would a Dockerfile come into picture? 18:57
melezhik I mean you can take out the logic 19:00
Packages , etc
Packages to be installed, sparky retaliation , etc 19:01
Installation 19:05
Some typos are funny 😄
patrickb I think I still don't understand.
melezhik Some typos are funny 😄 19:06
[Coke] it's running in a dockerfile now
in docker, that is.
I'm fine with it in a container, doesn't need a special VM. 19:07
patrickb What I did: I created a VM, installed a pretty naked Debian 13. Then installed git, podman, vim, tmux, (maybe some more). Then git cloned brownie and booted up the agent.
melezhik Ok, what ate you going to place there? This should be specific image not just with Rakudo but with running agent in it , 19:09
Ok. What are into to put into this VM image?
Ok, what ate you going to place there? This should be specific image not just with Rakudo but with running agent in it ,
Booting up brownie means some installing some dependencies, e.g. Sparky 19:12
Also means providing some env variables potentially
Booting up brownie means some installing some dependencies, e.g. Sparky 19:13
Also some specific Debian packages need to be installed as they required for some Raku modules tests 19:14
This all could be learned from Dockerfile
This is what I mean by “taking out from Dockerfile”
Sorry for duplicated lines, irc connection sometimes is terrible 19:15
patrickb but all of that is handled by the agent, right? The agent creates a container and performs all that install stuff in there, right? 19:16
The VM needs none of that. The VM of mine pretty much only has git and podman installed.
I'm logging off for today. o/ 19:19
disbot6 <melezhik.> It’s not a question of VM need something or not - VM is just a runtime method . So agent runs modules tests, process results and interact with orchestrator and send results to it and getting new requests from it . But agent does not provide initial setup and dependencies installations . It’s all carried out as process of building from Dockerfile in immutable fashion - build once - use many times . Of course this all 19:21
could be implemented inside agent but if we want to use container - it’s sort of antipattern for containers way
<melezhik.> However if for example VM runs agents as podman container which you just mentioned there is already Dockerfile from which is easy to build an image and run it inside VM. But in this case it’s defats idea of VM / bare metal installation , when we have one more layer via container 19:24
<melezhik.> Defeat 19:31
melezhik patrickb: - but that’s ok , I can create a standalone bootstrap script that would install all agents dependencies and run it on bare metal /vm machine as well - if this is something you want - please let me know , having two versions of installer could be a bit overhead but bearable , we can move core of an agent into distributable Raku module or something and handle VM vs container specific in tiny client … I am flexible 20:40
here
ab5tract I’m more concerned about there not being a corresponding apparatus for verifying the new payload. Maybe I’m off base but my impression is anything that knows I’m running an agent could push changes and restart the agent code
It’s in a container, which is good but not perfect in terms of protection 20:41
melezhik Anyways I am flexible in that case to and not tied to any specific runtime method , both VM and containers seem reasonable for me ) it’s not a big deal to support brownie pure VM installation as well ) 20:44
patrickb I find the current setup pretty nice actually. The VM gives me protection from malicious code and allows me to reset to a known state after every run (should I want that). The containers therein encapsulate the brownie components and automate their creation. 21:42
I see no benefit in trying to run things directly in the VM.
So from my side there is no need to change anything in that regard. 21:43
So to be clear, I don't need a standalone bootstrap script. The containers work just fine. 21:45
[Coke] melezhik: yes, please open tickets if brownie is pointing at module issues. 22:16
(completely missed that q earlier) 22:17
22:49 melezhik left

Powered by Cro and the Raku® Programming Language.

Please report any issues / comments / feature requests as an issue on App::Raku::Log.

Thank you!