01:34:17 Unstable branch on crawl.develz.org updated to: 0.28-a0-80-g9707f7362f (34) 01:54:11 Windows builds of master branch on crawl.develz.org updated to: 0.28-a0-80-g9707f7362f 02:12:24 <12ebering> Ok. Some serious thoughts re save compat breaking. 02:13:09 <12ebering> As I recall from the last time we were talking about this seriously, the big obstacle to bumping tag major is on the infrastructure side: the tag major bump handling automation for our trunk servers hasn't been tested since major 34 02:13:35 <12ebering> @gammafunk cdi is a full dgl set-up yes? so we could use that to test the transition 02:13:53 <12ebering> I understand a desire to switch to a rolling tag minor system but I think that's best done in a fresh tag major 02:15:37 <12ebering> tournament is tbh a good time to do it: we've been in recent contact with all server admins, players are re-directed to stable, and trunk is quiet 02:18:24 Unstable branch on cbro.berotato.org updated to: 0.28-a0-80-g9707f7362f (34) 02:20:17 <09gammafunk> I was actually arguing that in an earlier discussion about this with PF, yeah; the issue is TAG_MAJOR stuff that doesn't have an associated minor version, it's hard to clear that stuff out in a reasonable way except all at once 02:20:19 <12ebering> then, going forward, non-enum-reordering parts of save compat code can be conditioned on a TAG_MINOR_INVALID comparison and use aidanh's rolling system 02:21:15 <12ebering> yeah, we could in theory keep major 34 and use aidanh's code as is but that would require evaluating every TAG_MAJOR preprocessor check for which tag minor by hand 02:21:16 <12ebering> which 02:21:19 <12ebering> eugh 02:21:25 <09gammafunk> yeah, I guess the idea is that everything has to have a tag minor version though? 02:22:02 <12ebering> yes, but using 16 bits for tag minor we'll be hard pressed to use em all 02:22:22 <12ebering> also enum reordering still needs to be tag_major_filtered 02:22:28 <09gammafunk> is there a technical issue in terms of tag minor being save-level data and it not being accessible everywhere? 02:23:01 <12ebering> nope, because the constants are available in the same way TAG_MAJOR is 02:23:06 <12ebering> the binary knows only its own version 02:23:28 <09gammafunk> right, ok 02:24:42 <12ebering> anyhow, I'm not sure we want to rush this for this t, but perhaps we can get a plan and framework in place and do it next t 02:25:35 <09gammafunk> so if this rolling thing works like, if you have a save with mino ver less than minor_invalid, we refuse to load that save (since we've removed all associated save compat code) 02:28:03 <08wormsofcan> not sure how this got introduced, but exclusions in shoals leaks info about shallow water 02:28:15 <08wormsofcan> http://puu.sh/I1nc1/8db80e9b44.png 02:28:27 <08wormsofcan> all the blue tiles in this image are shallow water tiles 02:29:46 <08wormsofcan> I think it's tiles that are changed from shoal tides 02:31:47 <09gammafunk> hrm, never mind, I see why we still need the tag major for enum order changes even with rolling system 02:32:15 <09gammafunk> what we really need is to be able to see the new ordering and the old ordering at the same time and translate an old save's values into the new order (and then use the new ordering) 02:32:32 <08wormsofcan> http://puu.sh/I1neh/2c90b85c86.png 02:32:34 <09gammafunk> but it's non-trivial to do that 02:32:42 <08wormsofcan> this is how the leak looks after waiting with 5 02:34:44 <08wormsofcan> looks like this has existed since at least 0.25 02:40:04 <09gammafunk> @ebering regarding testing, yes, cdi is a viable testing location, but I think we'd want to do a test with CAO with an uploaded save. We also have to figure out how dgl scripts react to a non-transferable game and whether that needs some further script development. It might be fine in terms of it giving a "delete this save or just remain on your current version" prompt, but it might not work 02:40:04 properly. And admins will probably need the "force transfer" script to be able to detect and delete too-old saves. Not sure if dgl scripts currently have this facility for major/minor version, but it'll need to work for both going forward. Also cwz/lld/cpo are non-dgl, no idea what we can do about those. 02:41:03 <12ebering> doesn't cpo already force transfer 02:41:31 <12ebering> imo we should just set trunk to automatic force transfers, so that by the time a tag minor bump hits everything is already past the min tag minor 02:42:14 <12ebering> keeping trunk games playable in perpetuity is a bit wacky, we're not developing critical systems infrastructure with strong backwards compat requirements :d 02:51:37 <09gammafunk> yes it does force transfer, the problem isn't force transfer it's that when save compat break hits (and on the subsequent rolling ones), whatever scripts those servers have will be assuming they can force transfer a game. That will fail and render a user's trunk game not loadable at all 02:52:42 <09gammafunk> really those scripts just achieve force transfer by loading the save normally 02:52:58 <09gammafunk> dgl scripts just go out of their way to detect and prompt 02:53:13 <09gammafunk> and likewise have a facility to maintaining multiple versions of trunk 02:53:19 Monster database of master branch on crawl.develz.org updated to: 0.28-a0-80-g9707f7362f 02:53:43 <09gammafunk> so if a game can't be force transfered, dgl servers can serve up older versions of crawl 02:53:59 <09gammafunk> the nice part about this is it's seamless for users 02:59:16 <12ebering> so for the rolling compatibility I envison that when tag minor is bumped you don't bump it to the current value, but say, a tag minor that's 6 months old 02:59:40 <12ebering> so if everything has been regularly force-transferred it will remain seamless, even in the case of preposterous outages 02:59:51 <12ebering> agree that the tag major more complicated 03:00:07 <12ebering> that's why I included "recently in touch with all server admins" as a tournament-time pro 03:00:56 <09gammafunk> right, sounds like you're suggesting we have all dgl admins disable the prompting aspect of the dgl scripts and install a cron or something that regularly force transfers 03:01:10 <09gammafunk> which is doable but requires configuration changes on their part 03:01:44 <09gammafunk> it's not something where we can do something crawl/crawl repo side and affect this change 03:03:13 <12ebering> yes 03:03:18 <12ebering> that is exactly what I am suggesting 03:28:42 For tournament banners, is tier-1 prerequiste to getting tier-2? 03:28:52 Ie can I get Prophet II but not Prophet I? 03:29:44 <12ebering> There’s no prerequisite system and you get the points for the highest tier you earn only 03:30:29 Thanks 03:30:55 Fork (bcrawl) on crawl.kelbi.org updated to: 0.23-a0-4217-g7c68dc2372 06:25:07 Experimental (bcrawl) branch on underhound.eu updated to: 0.23-a0-4217-g7c68dc2372 07:37:35 Ditch Bison 2.3? https://github.com/crawl/crawl/issues/2031 07:38:28 And save PNG files properly. ;) 07:40:32 Things gotta move forward or this happens. https://en.wikipedia.org/wiki/Software_rot 07:44:45 Just a thought. Later Bison versions might have other benefits too. A simple warning can't justify the hassle. 07:48:37 The horizon is changing. For example, Python 2 support will be dropped soon from basically all the software projects. 07:50:28 So I'm not ranting for ranting's sake. 07:55:08 Take a note what's happening in this world. 08:19:49 <12ebering> They parted again 08:25:13 * Pinkbeast reads the issue and blinks 09:10:28 <06advil> Re save compat, something that I’m not sure gets remembered is that level data is not upgraded until the player enters the level, on the current implementation, so a force-upgraded save may require save compat code still when playing 09:59:35 Shafts aren't there when you search for "trap". 13https://crawl.develz.org/mantis/view.php?id=12623 by Yermak 10:04:10 <06advil> Also, would prefer not to make home brew (or whatever) a dep on Mac, it’s annoying that the provided version is so old but it works fine 10:04:49 <06advil> Not sure why that contributor is so fixated on this warning 10:11:58 <12ebering> That level loading feature was not something I had remembered, no. Hm. 10:14:10 <12ebering> Well, that goes on the list of things to look into 10:25:15 <12ebering> I said in irc that a version detection pr would be welcome but warnings like this are somewhat low priority but they might have parted. 10:26:11 <12ebering> Certainly agree that we should be able to build with the provided versions of one of the worlds largest Unix vendors 10:55:58 <10PleasingFungus> all this enthusiasm to break save compat 10:57:40 <12ebering> the tag minor counter counting up like a sword of Damocles 11:00:18 <10PleasingFungus> it's not tho 11:00:20 <10PleasingFungus> we covered this 11:00:33 <12ebering> no I know 11:00:46 <12ebering> that was a humorous remark! 11:00:54 <10PleasingFungus> immune to jokes 11:50:42 <06advil> One simple option might be to remove some of the more elaborate but old save compat fixes and warn the user, like we do for lava orc upgrades 11:52:02 <06advil> Personally I don’t care a lot about enum reordering, could drop that goal entirely as far as I’m concerned; it’s a permanent lost cause and things shouldn’t really rely on the order 11:53:37 <06advil> Doesn’t really address old level data on force upgraded saves either 12:11:23 <08nicolae> ditch save compat entirely, even within the same version 12:21:35 <10PleasingFungus> ditch saves. win in one session or quit 12:27:51 <09gammafunk> YO DID SOMEONE SAY BREAK SAVE COMPAT?! 12:32:30 <09gammafunk> I was wondering if there's some system you could use where you had access to both the current ordering and the new one so that re-ordering could be part of save compat fixes. You'd patch up the values as you loaded a save. I'm just not sure how that'd work in terms of implementation 12:58:48 <09gammafunk> oh crap, yeah I'd totally forgotten about that. How do you even deal with this? 12:59:22 <09gammafunk> do you have to have save compat iterate through every level to load/save? 12:59:34 <09gammafunk> seems like you might, which would make loading slow 13:07:39 <10PleasingFungus> follow your heart, obviously, but 13:08:15 <10PleasingFungus> unclear to me that save compat breakage has a reasonable effort to useful result ratio 13:08:36 <10PleasingFungus> like i just implemented a huge new feature for 0.28 over the last few days 13:08:45 <10PleasingFungus> that i think people will be real excited about 13:09:12 <10PleasingFungus> and breaking save compat seems like it’d be several times harder 13:11:55 <09gammafunk> I think we're all on the same page about wanting rolling save compat "break" (we probably need a simple phrase or term to describe the thing being proposed), but there's just disagreement about whether doing a save compat break before switching to that would be useful 13:12:27 <09gammafunk> specifically because there's the nasty issue of TAG_MAJOR_VERSION code with no associated minor version where we have no clear way to isolating that code 13:13:01 <09gammafunk> obv we can keep it in and handle each instance on a case by case basis, carefully matching any instance to some TAG_MINOR ourselves and then removing it 13:13:13 <09gammafunk> I'm guessing that'd be your proposal, PF? 13:13:54 <09gammafunk> since unminored save compat code living outside of tags.cc is the majority of save compat code, we defniitely need some way to phase it out 13:14:59 <09gammafunk> so far as I can tell, that's the only benefit a hard break actually gets us, just removing that unmatched code in one pass and not having to think about it, and we move to rolling save compat from that point 13:15:15 <09gammafunk> but it's true we could avoid any break at all if we were more careful 13:18:08 <08nicolae> speaking of breaks: let's say, hypothetically, i made a ghost vault for a branch that doesn't typically place them, like Slime. if i put DEPTH: Slime in the vault definition, would it show up, or would the predefined ghost vault lua functions make that not work. it looks like it would okay (assuming it got past the "gammafunk and kate" compilation step) 13:18:22 <08nicolae> i assume ghosts still place in Slime even if they never show up usually 13:19:10 <09gammafunk> it would work, but yeah it wouldn't use the CHANCE system we have for such vaults, since that depends on a statement and tag loaded by a function used by all current ghost vaults 13:19:24 <09gammafunk> I think the problem with slime ghost vault is that they're supposed to not place any loot 13:19:40 <09gammafunk> because slime vaults are not supposed to have loot, since you dive slime 13:19:52 <08nicolae> ah, there is that, yes 13:20:10 <08nicolae> here's the loot: a downstair 13:20:52 <08nicolae> eh, maybe i can put it in Lair at the slime generation depth, i noticed a few other ghost vaults themed around "poor bastard died on the way out of a portal" 13:21:27 <10PleasingFungus> my proposal is we don't bother with any of this until some very rainy day 13:21:52 <08nicolae> what is this huge new feature 😛 13:21:55 <10PleasingFungus> or unless there's some very compelling motivation i'm not aware of 13:22:05 <10PleasingFungus> no new features until after t! 13:22:22 <08nicolae> i know! but what is it going to be... 13:24:00 <09gammafunk> oh, so we have to sort of live with a lot of save compat code sitting around. I guess I don't love that because there's so much of it, but yeah I suppose it's not unreasonable. It is true it would give us time to think of some more elegant solution or just work on it iteratively 13:24:11 <08nicolae> also i have another quotes branch i'm working on, so heads up on that 13:25:11 <10PleasingFungus> the save compat code doesn't bother me, personally 13:26:16 <10PleasingFungus> really i'd just rather spend my time & energy on things to delight & torment players 13:27:53 <09gammafunk> yeah, so if I understand what you're saying, you're actually not really advocating for any kind of rolling break, we should just use aidanh's extention of minor version and presumably start tagging all new save compat stuff with a new minor version each time, but not actually phase out any old minor or major version until we're good and ready 13:28:12 <09gammafunk> I guess ideally we'd not do major ever since it's a hard break 13:28:16 <10PleasingFungus> yeah, basically 13:28:25 <10PleasingFungus> if we're going to do something, a rolling break would make the most sense to me 13:29:46 <09gammafunk> hearing that thing advil mentioned about level chunks being loaded at an arbitrary time after save transfer definitely adds a point in favor of what you're proposing; I think we'd have to restructure our level fixup code so there's some kind of pass being done over every level chunk every time a save is loaded 13:30:01 <09gammafunk> otherwise there's simply no way to deal with that under a rolling break system 13:33:22 <10PleasingFungus> it becomes sort of an exciting technical question, yeah 13:33:55 <10PleasingFungus> even if you did a major version break, you'd still run into the exact same issue after two new minor tags, right? 13:39:56 <09gammafunk> yes, with a rolling system you'd absolutely have to deal with this; what I'd imagine is you'd just do a fast pass over the save where you unmarshalled and remarshalled but without loading the data fully into memory 13:40:26 <09gammafunk> only concern is whether we'd in fact have enough info to run save compat code; not sure if it depends on level info only available if you actually load the level 13:40:32 <09gammafunk> probably not 13:44:48 <09gammafunk> unlike I had previously believed, thanks to aidanh's thing we can kick the can down the road as long as we like for any potential break, so all I'm going to do is test the transition from NUM_TAG_MINORS < 256 to NUM_TAG_MINORS >= 256 in a save I'll make and then upload to accounts on CAO and CDI 13:44:59 <09gammafunk> just make sure there's no breakage with e.g. dgl or something unforseen 13:46:08 <09gammafunk> oh I guess I'd have to set up a playable branch where this transition took place, but that's not hard to do 13:58:55 <10PleasingFungus> sounds like a fine plan 13:59:04 <10PleasingFungus> lmk if there's anything i can help with 14:19:40 <08nicolae> is there some way to use a player ghost for a different floor/branch, iirc as it stands now you can't get ghosts from volcanos and ice caves since there's no ghost vaults that generate there 14:22:45 <09gammafunk> no, that's a faciility we'd have to enable in the ghost saving code; I had plans to adapt various portals using a subvault system that would plug into all exsiting maps, where subvaulted ghost vaults would exist for each portal (potentially) 14:22:54 <08nicolae> i see, i see 14:22:55 <09gammafunk> and then the plan was to start saving portal ghosts 14:23:00 <09gammafunk> but I've not done this 14:23:02 <08nicolae> rip 14:23:19 <09gammafunk> I wonder though, would ghost with place work? 14:23:24 <09gammafunk> no idea if that works 14:23:37 <09gammafunk> I mean, to be clear 14:23:53 <09gammafunk> it won't work for place:volcano since there are no ghosts to load there (We're currently not saving them) 14:23:59 <09gammafunk> but I wonder if in general it would work 14:24:06 <09gammafunk> guess it's not relevant to your question though 14:24:58 <09gammafunk> it would be a fun little project to make ghost vaults subvaults for all the portals and then plug them in 14:25:27 <09gammafunk> you'd have to design subvaults themed for each portal, and I guess there's a sort of vague concern about clash of themes 14:25:46 <09gammafunk> do you make any additional monsters in these subvaults be very generic to the portal itself? does that matter? 14:26:26 <09gammafunk> arguably there are some portal maps where it might be tricky to find a subvault place 14:28:47 <09gammafunk> also pretty reasonable to just say that ghosts can stay in connected branches only and the added complexity of ghosts in places like portals might not be worth 14:30:55 <09gammafunk> @nicolae going back to your original question, all you have to keep in mind are the rules we have for ghosts: any ghosts should be behind runed door or transporter that is a no_tele_into area. For something like slime where we enforce no loot if not slime:$, you'd not want to put extra loot in the vault (which kind of makes any vault less fun for players) 14:31:01 <09gammafunk> also not sure that we save slime ghosts currently 14:31:43 <08nicolae> makes sense 14:31:55 <09gammafunk> ...huh 14:32:01 <09gammafunk> it seems we are saving various portal ghosts 14:32:15 <09gammafunk> cpp static const set ghosts_nosave = { BRANCH_ABYSS, BRANCH_WIZLAB, BRANCH_DESOLATION, BRANCH_TEMPLE, #if TAG_MAJOR_VERSION == 34 BRANCH_LABYRINTH, #endif }; 14:32:20 <08nicolae> yeah, i was reading the player ghost wiki page and it says "A player ghost is generated when your character dies, unless your character is undead or you die in one of the following areas: Dungeon level 1 or 2, the Ecumenical Temple, the Abyss, a Wizard Laboratory, or the Desolation of Salt[1]." 14:32:39 <09gammafunk> yeah so only places are wizlab/desolation 14:32:45 <09gammafunk> which is honestly pretty funny to me, but w/e 14:32:53 <08nicolae> and i was like... is that up to date? it says it's up to date for 0.24, which was after the ghost overhaul, so 14:33:13 <08nicolae> are there gauntlet subvaults with ghosts... 14:33:15 <09gammafunk> but I'm not sure it'd be wise to load ghosts from those currently unused ghost places 14:33:33 <09gammafunk> no 14:33:33 there are probably ghosts that get saved and not used 14:33:35 <09gammafunk> I thought about that 14:33:46 iirc there weren't really a lot of portal ghosts for most portals 14:33:58 <08nicolae> all those ghosts, stuck in limbo 14:34:22 <09gammafunk> seems like early portals would have a good number, but I'm not sure if something is preventing saving 14:34:40 <09hellmonk> One way to do ghost vaults in some icecaves is to put them in the shortcut where you placed the cloud generators 14:34:54 <09hellmonk> Instead of the cloud generator fight some ghosts 14:35:16 <09gammafunk> well I would just prefer them to follow the same logic we use in other places; completely optional standalone thing 14:35:31 <09hellmonk> Well you would runed door them off still 14:35:39 <09gammafunk> yeah, you'd use a subvaulting system 14:35:43 <09hellmonk> Just use that as the map segment to plug them in to 14:35:52 <09gammafunk> make a designated ghost subvault shape, plug that into every map 14:35:58 <09gammafunk> don't need a lot of space I think 14:36:02 <09gammafunk> and make subvaults 14:36:22 <09gammafunk> it would be fun and an extra layer to portals, but not sure it's really necessary 14:36:51 <09gammafunk> you could get fancy and have some special cases for specific maps if someone wanted to do that 14:37:29 <09gammafunk> like custom set of subvaults or w/e, but you'd pick some reasonably smallish rectangle shape 14:37:35 <09gammafunk> and fit that into every map somewhere 14:37:45 <09gammafunk> it is a bunch of work, but it's pretty doable 14:38:30 <09gammafunk> and it would be possible for a map to simply not support a ghost subvault if it wanted 14:38:50 <09gammafunk> would not load the ghost subvault setup function, basically 14:53:45 <06advil> One thing about this is that ghosts would be much better if they did something like pull by xl, rather than by place, but the ghost db is very place-structured unfortunately 14:54:18 <06advil> yeah, something like this should work in principle, not sure what would be involved in doing it 14:54:36 <06advil> basically just do an excursion to every level in the save 14:55:33 <09gammafunk> right, I just worry if you actually did that and fully loaded every level into the level data structures, it might be slow and noticeably impact resuming a game 14:55:42 <06advil> yes, probably 14:56:09 <06advil> it would need testing but it doesn't seem ideal to do that as a normal part of load 14:56:23 <06advil> this is where we need libcrawl as separate from crawl 14:56:47 <09gammafunk> ah, yeah 14:56:56 <09gammafunk> you could just do it outside of the game 15:09:16 <06advil> we need better webtiles moderation / user management tools 15:11:00 <10PleasingFungus> will 0.28 be the version with a roadmap for unified login? 15:11:34 <10PleasingFungus> a design doc? 15:11:42 <10PleasingFungus> a manifesto? 15:11:51 <10PleasingFungus> that’s a separate thing from what you mentioned 15:12:53 <06advil> whenever I think about this I come back to something more like federated login 15:13:21 <06advil> with some sort of meta-accounts that are a bit more centralized 15:14:23 <06advil> e.g. you have logins like advil@cbr2, advil@cao, and if you can verify those are all yours via some sort of key, they are treated as one by things that are aware of it 15:15:14 <06advil> it's much closer to what we have now than any sort of unified login, it would also provide a bridge for eventually delegating login to a "home" server and having it work on other servers, or something like that 15:16:46 <06advil> maybe it's not any simpler though 15:32:14 TuffENuff (L6 FoVM) Crash caused by signal #6: Aborted (D:4) 15:32:52 ZyklonBeast (L6 TrBe) Crash caused by signal #6: Aborted (D:4) 15:33:04 hm 15:46:49 looks ok, there's just a lot of people playing 16:32:02 <10PleasingFungus> hm, that might work? would avoid the issue of having to deal with merge conflicts between servers 16:32:12 <10PleasingFungus> when different people own, say, "ardl" on different servers 16:32:28 <10PleasingFungus> probably worth pursuing further 18:15:40 <06advil> perhaps one could use a more centralized version of that to merge "legacy" accounts as well 18:20:41 <10PleasingFungus> hm? 18:23:44 <06advil> rather than something actually federated, it could just be possible to use a token-based id scheme to link up existing accounts with a single new-style account, basically a more authenticated version of what !nick does 18:24:41 <06advil> anyways, dealing with existing accounts seems like one of the big things to resolve, and while "start from scratch" maybe is tempting, I don't think it's obviously the best or easiest thing to do 18:26:37 <10PleasingFungus> truth... what is 'truth' 18:26:46 <10PleasingFungus> who can we trust 19:15:09 you can trust me. reason: have '+' next to name in irc 19:45:19 <09gammafunk> @ebering are the instances of nonhep_wins here and line below a typo of nonhep_nonfe_wins? https://github.com/crawl/dcss_tourney/blob/0.27-tourney/database-views.sql#L194 19:45:52 <09gammafunk> got these errors locally when setting up the scripts, if they happened on cdo I must have missed them, but I don't recall getting any errors 19:46:37 <09gammafunk> the error being that there's no table nonhep_wins and the table creation sql likewise doesn't create this, so looked like a typo of the view name 21:55:23 -!- rt is now known as robin