|
This channel is intended for people just starting with the Raku Programming Language (raku.org). Logs are available at irclogs.raku.org/raku-beginner/live.html Set by lizmat on 8 June 2022. |
|||
|
02:05
ACfromTX joined
|
|||
| disbot2 | <ng0177> I ran a benchmark: GPU is about 4x faster on my hardware. time ./llamafile -ngl 999 <<EOF Translate the following German text into English: $(cat translate.txt) EOF i9:78s i9 RTX 3060: 17s | 06:41 | |
| <antononcube> Which model / llamafile ? | 07:12 | ||
| <ng0177> Mistral-7B-Instruct-v0.3.Q4_0.llamafile | 07:13 | ||
| <ng0177> Does an equivalent to #% chat gc prompt, model=gpt-4o, max-tokens=4096 with the accompanying directions exist for #% llama? | 08:16 | ||
| <ng0177> cdn.discordapp.com/attachments/768...f571e& | |||
|
11:12
soverysour joined
11:37
soverysour left
|
|||
| disbot2 | <antononcube> @ng0177 There are two types of LLM magic cells: 1. Direct access of LLM services providers - Each of them is for a “stateless” execution 2. Chat cells, each associated with a chat object - Those maintain “state”, - I.e. chats with them are “global” across the whole notebook | 12:28 | |
| <antononcube> The magic spec “#% llama ,,,” means direct access to a LLaMA model | 12:30 | ||
| <antononcube> A chat object cell magic can include provider and model specs. For example: #% chat per1, conf=gemini, model=gemini-2.0-flash, max-tokens=8192 | 12:36 | ||
| <antononcube> To use LLaMA models in the chat object cells “conf=llama” has to be specified. | |||
| <antononcube> If the LLaMA model URL is different than the default one the argument “base-url” can be used to specify it. | 12:37 | ||
| <antononcube> I was asked similar LLaMA questions before (by @rcmlz) -- I think I made a didicated notebook. I will find it and post it later today. | 12:48 | ||
|
13:09
soverysour joined
13:20
soverysour left
13:57
soverysour joined
14:14
soverysour left
14:36
lizmat left
14:37
lizmat joined
17:27
soverysour joined
17:40
soverysour left
18:29
soverysour joined,
soverysour left,
soverysour joined
18:42
soverysour left
18:56
soverysour joined,
soverysour left,
soverysour joined
19:16
soverysour left
23:17
lizmat left
23:18
lizmat joined
|
|||
