| whaack: | diana_coman: EOD Report: I missed the deadline for my article today. I worked on TheFleet all day again to try to get the bots online before I'm going to be away from my terminal. I obtained the max-chan-per-nick for my networks, setup join/part logging as well as kicked-from-chan logging, ^ switched the unique ids to pids + changed the db schema to have a specific fleetbotlogs table, created a deploy script, and then ran it. I ran | [01:29] |
| whaack: | out of RAM on my VM so I am only connected to 8 networks right now. I need to fix my script so that it takes good care to keep its memory usage minimal. There also seems to be an error with the reconnect functionality. | [01:29] |
| whaack: | and by 'connected to 8 networks' i mean i have 8 sbcl processes running, since a random few of them got cleaved due to running out of memory and since my disconnect logging is not working, i am not exactly certain which networks i'm connected to atm. | [01:32] |
| diana_coman: | whaack: well, a test of sorts it sounds like, at least; that out of memory thing sounds gross though. | [04:52] |
| diana_coman: | http://ossasepia.com/2020/04/21/ossasepia-logs-for-03-Jan-2020#1014544 - I can see that; otoh though, how would you order them? I can see the case perhaps for both UUIDs and pids but having just one still keeps a set of problems around. | [04:55] |
| ossabot: | Logged on 2020-01-03 23:05:12 trinque: I didn't want to be in the business of worrying about renumbering log lines if I merged logs from two sources, and had previously said all manner of things about that log-line by foreign-key reference | [04:55] |
| dorion_road: | http://ossasepia.com/2020/04/21/ossasepia-logs-for-03-Jan-2020#1014478 - it actually exceeded expectations on pretty much every front. I took a lot of pictures the past 2 days and will have an article to publish, so will spare details now. I disembark tomorrow morning local time, then have back to back red eye flights with stop over in las vegas and arrive to panama Monday morning. it'll be quite the relief | [08:11] |
| ossabot: | Logged on 2020-01-03 05:34:05 diana_coman: dorion_road: how's the cruise? | [08:11] |
| dorion_road: | to get my routine and rhythm in motion again. | [08:11] |
| diana_coman: | dorion_road: oh hey, sounds good; though on the article front, dunno, does it beat hunting then or what? | [09:14] |
| whaack: | diana_coman: re memory there is an easy fix i can do, right now each process unnecessarily keeps in memory the entire list of networks + all their channels. that is at least 5mb per process right from the start | [12:09] |
| diana_coman: | whaack: aha, no need to keep that in memory really, no. | [15:49] |
| trinque: | diana_coman: what problems do UUIDs cause? | [15:53] |
| trinque: | arguably they're a waste of space, in that all that might ever be said would fit in an int32 | [15:54] |
| trinque: | at any rate, what makes a good key for a row of data is an interesting problem | [15:56] |
| trinque: | I've long thought that eventually all this data in people's bots would be chattered among the bots to each other. | [15:57] |
| diana_coman: | trinque: I didn't say they were a waste a space, no; and ftr I can state it also: no, they are not a waste of space. | [15:57] |
| trinque: | just asking for your opinion of the problems they cause, not saying you're wrong. | [15:58] |
| diana_coman: | I just think that both UUIDs and PIDs are needed because they serve different purpose really | [15:58] |
| trinque: | sure, UUID doesn't tell you which you saw first, for instance | [15:58] |
| diana_coman: | trinque: exactly; and this matters for log lines because timestamp is such a mess that you can't rely on it. | [15:59] |
| diana_coman: | UUIDs otoh are useful precisely for identifiers though one can argue that you can always calculate on the fly some hash | [16:00] |
| trinque: | yep, hash is perhaps much more appropriate a (somewhat) universal reference | [16:00] |
| diana_coman: | the way I see it, as long as it's one db, might as well go with auto-increment since there's no loss as such; if/when mixing dbs, can identify same/different via a hash and otherwise if/when needed add perhaps another level of ref (e.g. provenance ) or otherwise still just calc and add hash at that time | [16:02] |
| trinque: | renumbering is quite the bitch, can be made easier by "on update cascade" | [16:03] |
| diana_coman: | anyways, I was honestly curious as to why you chose that (and I hadn't caught /recalled what you said re log-line by foreign-key ref http://ossasepia.com/2020/04/21/ossasepia-logs-for-03-Jan-2020#1014544 ) | [16:04] |
| ossabot: | Logged on 2020-01-03 23:05:12 trinque: I didn't want to be in the business of worrying about renumbering log lines if I merged logs from two sources, and had previously said all manner of things about that log-line by foreign-key reference | [16:04] |
| diana_coman: | yes, renumbering is a bitch; otoh database design has anyway some tensions as soon as the whole thing is worth anything so I doubt there's a one/single/ideal/perfect solution there. | [16:05] |
| trinque: | for sure, tradeoffs in all directions | [16:05] |
| whaack: | i will likely have to merge dbs further down the line in this project. i need to route through multiple ips to bypass the max-connections-per-host limit. i was originally planning to still have everything running on one server and just go through proxies, but i realize that trying to log absolutely all irc network activity is likely an exercise in DDOSing myself | [16:09] |
| diana_coman: | whaack: you need to produce first a clear report re all those numbers, you know? what decision can one make on this sort of shaky sands of there-is-this-limit and that-limit but dunno exactly . | [16:10] |
| diana_coman: | don't attempt to build the whole thing out of incremental blind adjustments; it'll end up a mess quite predictably. | [16:12] |
| diana_coman: | blind & undocumented* | [16:14] |
| diana_coman: | whaack: anyway, each chan should be in a single db though, right? I don't see why would a single chan end up split across several dbs so not sure there's much actual trouble merging otherwise. | [16:15] |
| whaack: | diana_coman: okay i will try to be more forward thinking with my design. i am still repeating that bad habit of "incremental adjustments" | [16:16] |
| whaack: | diana_coman: yes a single channel would never be split accross dbs | [16:17] |
| diana_coman: | whaack: hm, did that MIT ever mention development methodologies at all? lolz | [16:20] |
| diana_coman: | whaack: anyway, if you have a use case for those UUIDs, you should have argued it and kept them in; if you don't, then they are not in. | [16:21] |
| whaack: | diana_coman: i blame myself for not taking advantage of MIT. but the default software system design class that I took was a how-to-java class | [16:21] |
| diana_coman: | whaack: there is some space and even need for incremental, sure, but it's not all-there-is. | [16:22] |
| diana_coman: | so then programming language class was what? | [16:22] |
| whaack: | diana_coman: what do you mean programming language class? I didn't take any class on language design or any class that *intended* to teach to a specific language | [16:25] |
| diana_coman: | whaack: software system design means > how-to-any-language | [16:26] |
| diana_coman: | ie by the name it actually intended to teach something other than a language; and otherwise hm, no..introductory programming course or something lower level (that I could see focusing on how-to-x-language)? | [16:27] |
| whaack: | http://stellar.mit.edu/S/course/6/fa14/6.005/materials.html "Software Construction" was the name of the class | [16:28] |
| diana_coman: | ahahaha; why not "nuts and bolts that java holds" | [16:29] |
| diana_coman: | whaack: anyways, I guess I still haven't fully grokked the current MIT; don't take it to heart. | [16:32] |
| whaack: | diana_coman: thanks lol. I would not major in CS if I went there again, my advisor warned me against it and i did not listen. | [16:34] |
| diana_coman: | whaack: what did they advise for? | [16:34] |
| diana_coman: | "against it" is easier than "for it", lolz. | [16:35] |
| whaack: | diana_coman: He said anything else I was interested in. His argument was not that the CS curriculum was bad, but that I would probably learn CS on my own time, and that I wouldn't be taking full advantage of resources at mit by majoring in cs. (i.e. why not major in mechanical engineering and learn how to use a mil lathe?) | [16:37] |
| diana_coman: | good point really; so... why didn't you? | [16:41] |
| whaack: | diana_coman: I had decided that for sure I wanted to major in cs and stubbornly closed myself to his input | [16:42] |
| diana_coman: | (normally there would have been a retort to that "point" of your advisor but possibly not by that time anymore so they were right either because they knew CS well enough or because they didn't know it well enough, lol) | [16:43] |
| diana_coman: | ah, so ~the mech eng is not fashionable enough and lathe is not lisp. | [16:44] |
| whaack: | the intro class freshman year was also fun and imo well run. we programmed little vehicles that used sonar sensors to get information about their surroundings to solve mazes | [16:45] |
| whaack: | diana_coman: ^ pretty much | [16:45] |
| diana_coman: | btw the moment one says that "you'll learn X on your own time", that *means* the curriculum is bad; because if you can indeed learn it just the same on your own time then sure, wtf "university" | [16:45] |
| diana_coman: | and the sane retort to what I suspect is his different background simply rather than much more than that would have been that ahem, you can also learn to use mil lathe on your employer's money later on as needed but you'll never get access again to far more valuable resources such as Minsky, | [16:47] |
| diana_coman: | as in "working with Minsky"; but myeah. | [16:47] |
| whaack: | they do a career fair at the beginning of the year too, and it's a big event for the SV circus show, all the cs majors are getting 5-0/hour internships for the summer while the other majors generally struggle to find jobs | [16:48] |
| diana_coman: | I can imagine. | [16:50] |
| diana_coman: | lobbes: re your search for bugs, the whole discussion can easily go through CS, Maths, Logics and Philosophy in a single paragraph too, lol; might fit better in a conversation or as an article if you want to look at that bit in more depth. | [17:07] |
| lobbes: | http://younghands.club/2019/12/24/ejb-plan-week-2-jan-2-jan-5/#comment-261 << diana_coman: I've not really ever tested in an automated fashion before. I just kinda manually try to break things as I go (and really even then I'm not too organized in exactly what to be testing for). | [20:38] |
| lobbes: | diana_coman: I can definitely see how this can be an in depth discussion. You know, I think we have at least two items in a conversation queue now, so I wonder if we oughta have like a standing meeting at a certain interval? | [20:38] |
| whaack: | diana_coman: I have my bot running now connected to 7 networks with a more robust reconnect feature. I am capped at 7 because of memory issues - the problem is I load quicklisp in my environment which by itself is ~70MB. So a next step is to figure out how to load my dependencies without quicklisp. | [22:17] |
| whaack: | actually i may be able to use asdf:load-system instead, i see that in trinque's ircbot | [22:24] |
| whaack: | err in trinque's ircbot USAGE file | [22:24] |
| whaack: | trinque: I think I may understand this if you never got around to it. The thread that normally calls ircbot-reconnect is the ircbot-ping-thread. However the ping-thread commits suicide because ircbot-disconnect calls (sb-thread:terminate-thread (ircbot-ping-thread bot)) . So when the ping thread calls ircbot-reconnect after calling ircbot-disconnect it never gets to the ir | [22:56] |
| ossabot: | (trilema) 2019-02-07 trinque: now that we're at something of a cuntoo interude while folks are testing, I'll find some time to figure out what deedbot gags on time to time re: reconnecting | [22:56] |
| whaack: | cbot-connect or ircbot-connect-thread. If you test ircbot-reconnect (in the repl or such) you will not see the bug because then ircbot-reconnect is being run from a different thread. | [22:56] |
Comments feed: RSS 2.0