Minutes of the SIPB Meeting of 2012-02-13 The meeting was called to order by dwilson at 19:30. In attendance were Voting members: phurst ezyang achernya bbaren jhamrick afarrell | adehnert Associate members: tlyu asedeno jhawk mats_a | kenta lgdean Prospectives: omalley1 dlaw steb | bsw Guests: khobbs Administrivia ============= dwilson: sipb-ec elections are next week. Be sure to attend. Read the constitution if you're curious about the details, or talk to the current EC. Think about if you want to run, but there is no campaigning in advance. Project Updates =============== jhawk: sipb-afs had an outage on Sunday, affecting the whole SIPB AFS cell 6AM to 1:20PM. The big takeaway is that no one reported the outage, and there was some confusion whether or not maintainers were really aware of it. But if someone had sent mail to sipb-afs, that would have been particularly helpful. Also, the sipb-afs maintainers need to look at documenting more immediate escalation paths, because several people were not reading email. There was some sort of bizarre disk failure that managed to not be covered by RAID. Processes trying to access that RAID device hung, including the AFS service process; the AFS server was half hung, clients thought it was hung, but not that hung. So, two can't happen scenarios caused the AFS cell to be down for the morning. There were some hardware issues. sipb-afs is going to think about how to better publish their escalation path. dwilson: Do you need new disks? jhawk: I don't think it's pressing. tlyu: There was a report of a professor demanding an explanation for why the dialups were wedged on Sunday morning, on -c consult. kenta: Do you know if there was data loss? jhawk: No known data loss. steb: I'm working a mobile client for zephyr with bsw. We need people to submit feature requests! bsw: It's for 21W.789. Zephyr to -c zmobile -i request kenta: There's a Kerberos implementation for Android? steb: No, we're writing a server; potentially a plugin for BarnOwl, which the mobile app will talk to. bsw: It's supposed to be a mobile connection which is better than an SSH terminal. SSH is very clunky; we want something easy to use. steb: It would also allow other people to write clients, which interface with Zephyr, without needing to deal with the protocol. bsw: We'd like to talk to the Linerva maintainers about how this would work. achernya: Note that Linerva is probably going away. tlyu: It sounds like you're trying to make a proxy server of sorts for zephyr. Do you intend to make a long running hosting service for this? steb: We're planning on doing it the way the current BarnOwl service works; so it's one BarnOwl for ever server. bbaren: I'm pretty sure there's a Kerberos implementation for Android... steb: But that won't work. dwilson: There are people in the room who know about Kerberos. achernya: A few days ago, k-s, the current slave in our master-master setup, reported that there was a replication failure. This was due to a failed query on slave, because the contents of a particular table had diverged between the two replicas. This was due to a user's use of the UUID function, which because we have statement-based replication, caused different UUIDs to be stored in each table. When the user made a table structure change, in this case adding a UNIQUE key, it failed on the slave because there were duplicates in the field. We went through and found the problem and corrected replication. And then, we went on a quest to find a generic solution for this problem. We've been running a tool from Percona (a commercial company that supports MySQL) which checksums tables, and we're looking for divergent tables. If we are not able to do so, we'll have to schedule downtime to synchronize kitchen-sink from sliced-bread's data. If we were to do this, we'd also p-k mixed mode, which will use row-based replication for unsafe queries, and statement-based for everything else. Unfortunately, MySQL doesn't ship mixed as default. We're not sure they reverted; we think it's because they wanted to support the abuse of servers, because no users know we're replicating; but if there are performance issues tlyu: Is there any automated way to unwind the divergence? achernya: This is just the particular user's case. There is also a sync tool neat current master set rows back to themselves, which then get replicated to the slave and execute and create. But this requires an index be present on the table you run it on. Other ===== Other Other ========== adehnert: Tech Squares has its first class tomorrow. If you like Square Dancing, you should try it tomorrow! There are SIPB members who do tech squares. 8PM Lobdell meeting adjourned at 19:41. Minutes taken and submitted by ezyang.