01:34:24 Unstable branch on crawl.develz.org updated to: 0.26-a0-541-g3b3d21e777 (34) 01:55:07 Windows builds of master branch on crawl.develz.org updated to: 0.26-a0-541-g3b3d21e777 02:09:59 -!- amalloy is now known as amalloy_ 02:53:25 Haunted (L17 GrFi) ASSERT(range >= 0) in 'beam.cc' at line 601 failed. (beam 'fireball', source 'tengu reaver', item 'none'; has range -1) (Zot (ZotDef)) 03:15:38 Unstable branch on crawl.beRotato.org updated to: 0.26-a0-541-g3b3d21e (34) 03:30:58 Fork (bcrawl) on crawl.kelbi.org updated to: 0.23-a0-3545-g7c84a495e7 03:39:27 advil: the vbuf that viewwindow renders does include all tiles on the screen, including those outside of the max-los square 03:40:37 i haven't profiled viewwindow much, but i think the best way forward would be to add caching within viewwindow, rather than rely on callers to know when to call it 04:49:06 hm, for repeated disaster areas, around 65% of viewwindow time is spent looking for overlay cells to add 04:49:13 !source view.cc:1656 04:49:13 https://github.com/crawl/crawl/blob/master/crawl-ref/source/view.cc#L1656 04:50:12 time spent there will scale linearly with the number of overlays added on the screen; disaster area at high piety is probably the worst 04:50:43 and also scale linearly with the number of tiles on the screen (including out-of-los) 04:51:09 ah, yes, a megazig would be quite bad for this 05:20:25 Delver + Pikel interaction 13https://crawl.develz.org/mantis/view.php?id=12346 by Le_Nerd 05:50:09 New branch created: optimize-view-overlays (1 commit) 13https://github.com/crawl/crawl/tree/optimize-view-overlays 05:50:09 03Aidan Holm02 07[optimize-view-overlays] * 0.26-a0-542-gbaa144b: Improve viewwindow performance when rendering animations (advil) 10(10 minutes ago, 1 file, 66+ 14-) 13https://github.com/crawl/crawl/commit/baa144ba5c2e 09:34:26 ??hilarious deaths 09:34:26 hilarious deaths[1/152]: For funny deaths that don't belong in {embarrassing_deaths}. Xom thinks this is hilarious! Also, anyone adding bigterm games to hilarious_deaths should be shot. If a lot of entries look missing, see {hilarious_cdo_deaths}. 09:34:37 ??hilarious deaths[2 09:34:37 lich pack[1/1]: !tv * place=D:7 killer=lich 1 09:37:21 is footv just dead? termcast.develz.org doesn't work on either telnet or http 09:51:15 03Aidan Holm02 07* 0.26-a0-542-gbaa144b: Improve viewwindow performance when rendering animations (advil) 10(4 hours ago, 1 file, 66+ 14-) 13https://github.com/crawl/crawl/commit/baa144ba5c2e 09:51:40 aidanh: it's been dead since Sequell was moved, and I haven't bothered snark about it, but I'm going to contact him soon about a few isues with sequell 09:51:40 in the mean time your best best is to download ttyrec and use ipbt 09:52:08 gammafunk: thanks for the info; i was actually just wondering about what it would take to move ttyrec storage to a centralized place 09:52:24 since i remember footv is the main client that actually downloads them 09:52:44 aidanh: yeah, ttyrec storage has been a pita for years, so something like that would be cool 09:53:25 footv also uses "timestamp" files (ending with suffix .ts) to match game turns to a time in the ttyrec, so those have to be handled as well 09:53:40 so that you can e.g query a milestone and request a TV of it 09:53:47 hmm, and i think milestone files came into the mix too? 09:53:55 ah, that's the timestamp files then 09:54:10 tbh I have no idea how those .ts files are processed 09:54:16 something I should figure out at some point 09:54:42 i think the potentially bigger issue would be that ttyrec files are on the host for the duration of the game, but move afterwards 09:54:48 as for moving ttyrecs to somewhere central though, how would server admins copy the files? and how would you deal with space requirements? 09:54:53 yep 09:54:57 although can people even watch games-in-progress with footv anyway? 09:55:25 i was thinking storage on backblaze, with a $5 droplet that proxies access to various buckets 09:55:39 I believe they can in that they could watch ttyrec actively written to by querying a milestone, but I don't know that they could follow it via footv 09:55:48 obviously you'd just spectate the game anyhow 09:55:51 if you wanted that 09:56:08 at least one server puts ttyrecs in a weird dir and moves them later 09:56:23 and I think most servers have a cron job to compress ttyrecs 09:56:29 that's part of the dgl scripts even 09:57:40 it might be easier to just not support querying games in progress, if it's too complicated for the value it would give 09:59:16 re moving to central storage, a script would post ttyrecs to the ttyrec server, and it could compress them and then upload them to backblaze 10:00:16 as for space requirements, I have a backblaze account with 10G free storage that I'm not planning to use; should be possible to split ttyrec storage across multiple accounts, depending on requirements 10:02:12 Unstable branch on crawl.kelbi.org updated to: 0.26-a0-542-gbaa144ba5c (34) 10:04:31 shallott was already a centralized place for ttyrecs for some servers, though I'm not sure if that's still happening (or can be assumed to happen forever) 10:04:46 aidanh are you aware that I've already been down the road of whether to move ttyrecs to a b2 bucket? 10:05:00 advil: i guess i'm not 10:05:29 I did actually move some, the highest-producing cao accounts (which are both bots) are on my b2 bucket 10:05:55 cao has something like 800G of ttyrecs 10:06:51 automatically moved? how does that work? 10:07:05 yeah, some filtering mechanism would definitely be needed 10:07:08 well, I wrote scripts that would automatically move them 10:07:21 not super fancy 10:07:31 there are already better scripts for moving ttyrecs to shallott I believe 10:08:16 the fact that it's structured into a really large amount of small falls makes it a pain to transfer, and while cheap to store leads to super high class c transaction costs once on your b2 account 10:08:16 I have around 8GB of ttyrecs on mine and that's already enough that listing all files in the bucket costs something like $0.01 10:08:28 or it's close to that at least 10:09:15 i was planning to keep a sqlite database on the $5 droplet to mitigate that 10:09:42 *small files 10:10:35 at this point I started looking into how many ttyrecs actually get viewed 10:10:48 and the answer was, multiple orders of magnitude lower than exist 10:11:40 (unfortunately that data was lost in the sequell reset afaict, so all we have are the queries in the logs here) 10:11:55 anyways, I don't think we should keep ttyrecs 10:12:23 it's a waste of dev/admin time and effort 10:12:27 it certainly wasted mine 10:13:14 yeah, i remember sequell or something kept records of which files were accessed; that's what got lost? 10:13:48 yeah, I think the view counts got reset 10:13:48 damn 10:14:16 i don't think there's much value in storing the vast majority of ttyrecs either, but if they're going to be stored it's worth doing properly 10:14:33 ...are there any servers that don't store them? 10:14:37 yes, cpo 10:14:41 er 10:14:42 wait 10:14:45 no that's not true 10:14:57 I think alex was talking about not storing them 10:15:04 but yeah all servers do currently save ttyrecs 10:15:35 for cwz, it doesn't have proper hookup to footv (even when that was working) due to how it stores ttyrecs in a different dir from the .ts files 10:16:07 the problem for me right now is that if they are *not* stored it's worth doing it properly (which is to say not a tech problem, but community consultations, letting people download their own, deciding on appropriate deletion policies, etc.) 10:16:50 yeah, you were saying something like storing them for...was it a year? 10:17:22 a big part of the benefit of centralized storage is centralized policy 10:17:35 it's possible that people from the community would step up to help with storage, but really the only person who I've heard expressing interest in that is minmay 10:17:55 right, I know minmay actually has cszo ttyrecs 10:17:56 the percentage of crawl community that watches them is quite small 10:18:15 I think she was sounding me out on whether I'd host them 10:18:37 (stupid question, what *are* minmay's pronouns? I've seen a lot of variation on this..) 10:18:45 it's she 10:19:19 ok 10:19:51 I do expect if I announced a plan to delete them we'd get people trying to scrape cao 10:19:58 heh 10:20:12 uploading them to b2 is prohibitively slow also btw because of the many small files problem 10:20:34 have you tried uploading in parallel? 10:21:05 to whatever small extent cao allows 10:21:32 maybe the s3 api would improve this? 10:22:48 is the speed a bottleneck in practice? i.e. are ttyrec write speeds really faster than b2 upload speeds? 10:23:17 i gave up with the b2 s3-compatible API after I couldn't get pre-signed posts to work 10:23:27 at least the plain b2 API has python examples 10:34:14 yeah, I would guess that's not an issue, but I was estimating weeks to months of time uploading to b2 if I wanted to move everything off of cao 10:35:06 btw just to connect the dots, cao doesn't have a python version that can handle b2, but this is one thing I used your chroot setup for 10:36:38 the other consideration is this: once you have a huge lump of ttyrecs on a cloud storage site, it becomes much harder to do any reorganization 10:37:04 if I were to do this properly I'd rather start from the ground up with the file format itself 10:37:16 ttryecs would be *much* more tractable if they were coherently grouped by game 10:37:54 so a very simple metaformat would be a tarbell that is labeled using a morgue-like file naming scheme, and has all .ttyrec and .ts files for that game 10:38:29 glad to see the chroot setup is coming in handy for something :) 10:38:49 of course this has downstream consequences for footv if that ever comes back 10:39:04 yeah, i was thinking tarballs of all related files would be much more friendly 10:42:15 !lg * year ntv>0 10:42:17 22. ISplatSpriggans the Intangible (L26 SpDe of Elyvilon), escaped with the Orb and 3 runes on 2020-08-11 20:52:54, with 1228916 points after 151893 turns and 7:02:20. 10:42:36 that's the number of ttyrec views since the sequell reset 10:42:45 the 22? 10:42:48 yeah 10:42:58 (that number was 1429 when someone ran the query earlier this year) 10:43:16 !lg * ntv>0 10:43:23 24. ISplatSpriggans the Intangible (L26 SpDe of Elyvilon), escaped with the Orb and 3 runes on 2020-08-11 20:52:54, with 1228916 points after 151893 turns and 7:02:20. 10:43:30 hm, not sure why those aren't the same 10:43:53 ... does it increment when you query it? 10:44:03 !lg * ntv>0 10:44:10 24. ISplatSpriggans the Intangible (L26 SpDe of Elyvilon), escaped with the Orb and 3 runes on 2020-08-11 20:52:54, with 1228916 points after 151893 turns and 7:02:20. 10:44:14 ..huh 10:44:20 anyways, once I started seeing these numbers, all this complicated effort seemed just not worth it 10:44:27 yeah, agreed 10:44:49 !lg * !year ntv>0 10:44:50 if we're going to centralize ttyrec storage, it may be wiser to just cut our losses with existing files 10:44:55 2. Eronarn the Necromancer (L27 DsNe of Sif Muna), collapsed under their own weight caused by gaining the bat transformation on Lair:2 on 2008-08-27 07:49:39, with 538692 points after 150318 turns and 17:21:18. 10:45:07 what a way to go 10:45:07 oh 10:45:15 maybe `year` is a constraint on the game, not the view 10:46:06 I suppose that looking at these numbers is a bit unfair given that footv has been broken since then anyways 10:46:59 but the total number of ttyrect views that sequell had recorded at the point where these queries were run in the logs was 34178 10:46:59 !lg * cao 10:46:59 3554657. HunfridMar the Caller (L1 MuSu), slain by a kobold (a +0 dagger) on D:1 on 2020-08-16 14:44:18, with 1 point after 416 turns and 0:00:47. 10:47:34 where cao alone has 3.5 million ttyrecs (or close to it, I think in the early days some got deleted as disked filled up) 10:49:32 thanks for the viewwindow fix btw 10:51:45 for profiling, another thing you can try is holding tab with autofire -- there's a lot of cost there in trunk that's not about the animations (I have a branch that deals with this) but you can definitely see the animations 10:52:31 I suppose alternatively you could temp revert e645bd13c to exaggerate the problem 10:52:31 %git e645bd13c 10:52:31 07advil02 * 0.26-a0-450-ge645bd1: Retune qazlal animations 10(3 weeks ago, 3 files, 31+ 13-) 13https://github.com/crawl/crawl/commit/e645bd13c135 10:54:22 ah, good idea 10:56:07 at least on local tiles, more time is now spent repacking opengl buffers than inside viewwindow 10:56:26 !kw year 10:56:27 Keyword: year => end>${now()-interval('365d')} 10:57:05 yeah, I don't know why I thought that would have anything to do with the ntv part 11:44:43 gammafunk: http://crawl.akrasiac.org/rawdata/gammafunk/ only a handful of your ttyrecs have ts files; is that expected? 11:46:43 that's one per game there 11:47:50 so there are multiple ttyrecs per game? 11:47:57 yes, ttyrecs go by login session (or something like that) 11:48:18 !lg . -ttyrec 11:48:20 563. advil, XL27 HONe, T:99500: Can't find ttyrec 11:48:32 !lg . -2 -ttyrec 11:48:33 562/563. advil, XL27 GnWr, T:123992: Can't find ttyrec 11:48:35 hm 11:48:36 well 11:48:50 hm, that doesn't seem right; there's a ttyrec for 2014-02, but the earliest 2014 ts file is for 2014-06 11:49:47 !lg gammafunk vpcj cao -ttyrec 11:49:49 1. gammafunk, XL5 VpCj, T:7249: http://crawl.akrasiac.org/rawdata/gammafunk/2020-06-16.00:17:37.ttyrec.bz2 11:49:49 unless the ts file is created only when the game ends 11:51:29 oh wrong one 11:52:15 I'm also not sure if a ts file will get written for a 0 turn quit 11:52:47 which one of the 2014 games is 11:53:34 !lg gammafunk cao desu -ttyrec 11:53:34 No games for gammafunk (cao desu). 11:53:45 !lg gammafunk cao desu wizard -ttyrec 11:53:46 No games for gammafunk (cao desu wizard). 11:54:13 !lg gammafunk cao hufi -ttyrec 11:54:14 2. gammafunk, XL1 HuFi, T:0: http://crawl.akrasiac.org/rawdata/gammafunk/2015-03-22.18:26:29.ttyrec.bz2 11:54:54 !lg gammafunk cao desu wiz -ttyrec 11:54:54 No games for gammafunk (cao desu wiz). 11:55:03 well, not sure why that one doesn't exist at all 11:56:02 sequell does have some code somewhere for stitching all this stuff together 11:56:14 gods... 11:56:39 the .ts files are used in the seek operation as I understand it (since ttyrec as a format makes seeking nightmarish) 11:57:00 what I was really looking for is a game record with multiple ttyrecs 11:57:08 !lg . won 0.20 -ttyrec 11:57:09 1. advil, XL27 TrMo, T:80393: Can't find ttyrec 11:57:26 something's up with cbro ttyrec lookup I guess 11:57:31 !lg . won cao -ttyrec 11:57:33 4. advil, XL25 DDHe, T:76751: http://crawl.akrasiac.org/rawdata/advil/ 2014-08-23.17:32:22.ttyrec.bz2 2014-08-24.20:03:40.ttyrec.bz2 2014-08-27.16:07:59.ttyrec.bz2 2014-08-27.17:55:24.ttyrec.bz2 2014-08-28.13:17:52.ttyrec.bz2 2014-08-28.13:17:58.ttyrec.bz2 2014-08-29.02:07:38.ttyrec.bz2 2014-09-21.19:39:08.ttyrec.bz2 2014-10-05.22:24:04.ttyrec.bz2 11:57:37 there we go 11:58:20 aha 11:58:55 !lg . won cao x=gid 11:58:55 4. [game_key=advil:cao:20140723173238S] advil the Bringer of Life (L25 DDHe of Elyvilon), escaped with the Orb and 3 runes on 2014-10-06 00:05:00, with 1573716 points after 76751 turns and 14:11:45. 11:59:11 does your bot-ttyrec uploader handle any of this, by any chance? 11:59:32 no 11:59:49 it would have to wait until each session is over at least, no? 12:00:00 so as not to move a ttyrec being appended to 12:00:15 it doesn't handle games in progress at all, I've only used it to move archival ttyrecs 12:00:25 it's just a simple modification of the .bz2 script (that compresses ttyrecs) 12:01:17 actually that script may have something that checks if a file is open, 12:01:52 yep 12:01:55 uses lsof 12:02:08 compress-ttyrecs.sh? 12:02:23 it's a cao-specific one, not actually sure if it's used elsewhere 12:02:54 naturally 12:04:27 still, compress-ttyrecs.sh could probably be adapted to upload to b2 via a ttyrec server 12:04:46 here it is if you want to see: https://gist.github.com/rawlins/96f03df884b1a049a3d5a263983b300e 12:04:52 it'd be nice to group ttyrecs and timestamp files from the same game though 12:04:59 thanks 12:05:50 like I noted there, was really only a proof of concept, I'm not sure it's suitable to run except on a per-user basis as is 12:06:46 hm, what about serving downloads? i guess if it's ttyrecs older than 2018 that's not really a concern though 12:07:00 so it's all accumulated ttyrecs for now I'm guessing 12:07:07 you mean if someone is downloading as it happens? 12:07:39 yes, my original plan was to bulk archive everything on cao older than 2018 12:07:40 the lsof should avoid that, although it's racy of course 12:08:09 yes, I wasn't really sure how well that lsof thing would work in practice, it's just what the .bz2 script used 12:08:12 i meant after upload and removal, http requests will return 404; do you have any mechanism to redirect to a b2 download URL? 12:08:22 oh, no that needs to be handled in sequell 12:08:24 which it can be 12:08:30 but I never bothered for these games tbh 12:08:43 fair enough 12:08:44 what I archived is two bots, qw and tstbtto 12:09:03 if I'd continued I would have 12:09:12 guess i should finally clone sequell 12:09:58 it seems to not be working but cbro's entry in sequell config is a good example of how to do this, in that cbro ttyrecs are spread across cbro itself for the newest ones, and shallott 12:09:58 I believe other servers use something else, maybe cpo is using s3 12:09:58 wait, ruby *and* perl? 12:10:02 lol 12:10:08 why are you doing this to yourself :-P 12:10:22 sometimes i wonder 12:10:39 I do really think ttyrecs are not worth this level of effort in the end, that's why I stopped 12:11:28 there's a bunch of Very Serious questions that people involved in libraries, archives, art conservation, etc., think about all the time. My partner is in fact a professional art conservator, and I talked to her a while about this 12:11:50 the fact is that in the world you can't archive everything 12:12:01 or conserve everything, etc 12:12:24 the volume of "artifacts" (where in some digital preservation sense these would be included) is just vastly higher than the amount of human effort that can be directed at them 12:13:34 this has already been true at every point in human history, but it's currently the worst it's ever been 12:14:26 definitely true, and the value of most artifacts would either be so low as to not be worth the effort to begin with, or would depreciate rapidly 12:25:04 hm, yeah, it looks like sequell lists all ttyrecs in the users' folder and then filters by timestamps from the corresponding milestone entry 12:27:56 aidanh (L1 MiFi) ERROR in 'wizard.cc' at line 78: Intentional crash (D:1) 12:28:10 !crashlog 12:28:11 22518. aidanh, XL1 MiFi, T:51 (milestone): http://crawl.akrasiac.org/rawdata/aidanh/crash-aidanh-20200816-162756.txt 13:06:09 Unstable branch on crawl.akrasiac.org updated to: 0.26-a0-542-gbaa144b (34) 14:25:54 -!- amalloy_ is now known as amalloy 15:08:54 this is a bit old but I just ran across some fairly quality poetry about dcss and the devteam: https://www.reddit.com/r/dcsscirclejerk/comments/gklybh/has_anyone_seen_that_bogus_knockoff/fqrxcdm/?utm_source=share&utm_medium=web2x 15:09:33 (I guess the url text prepares you for the context) 15:31:44 advil: yeah, I remember when that was posted; I like to think they picked my name because I was probably the most frequent target of MR trolling drama that post was generally referencing and not just because it happened to rhyme 15:32:38 and I'll not ask by what means you recently stumbled upon that comment 16:01:09 <|amethyst> ??|amethyst[23] 16:01:10 |amethyst[23/24]: <|amethyst> If it's a buff <|amethyst> That you desire <|amethyst> That's too tough <|amethyst> Die in a fire <|amethyst> Burma-Shave 16:50:40 -!- amalloy is now known as amalloy_ 18:24:32 Unstable branch on underhound.eu updated to: 0.26-a0-542-gbaa144ba5c (34) 18:45:35 advil: you inspired me, I've just changed CPO policies to keep replays for only 3 months 21:30:37 -!- amalloy_ is now known as amalloy 23:42:34 turnerjer (L8 HaCK) ASSERT(mons_is_mons_class(mons, MONS_DOWAN)) in 'mon-death.cc' at line 3238 failed. (D:6)