| ↑back Search ←Prev date Next date→ Show only urls | (Click on time to select a line by its url) |
| Who | What | When |
|---|---|---|
| *** | erratic has quit IRC (Excess Flood) | [00:59] |
| .... (idle for 19mn) | ||
| erratic has joined #arpnetworks | [01:18] | |
| .................................. (idle for 2h49mn) | ||
| erratic has quit IRC (Excess Flood) | [04:07] | |
| erratic has joined #arpnetworks | [04:14] | |
| .................... (idle for 1h39mn) | ||
| perlgod has left "WeeChat 1.9.1" | [05:53] | |
| erratic has quit IRC (Excess Flood) | [06:06] | |
| erratic has joined #arpnetworks
ziyourenxiang has joined #arpnetworks | [06:14] | |
| ....... (idle for 33mn) | ||
| mloza has joined #arpnetworks | [06:50] | |
| ........................ (idle for 1h59mn) | ||
| erratic has quit IRC (Excess Flood) | [08:49] | |
| erratic has joined #arpnetworks | [08:57] | |
| ..... (idle for 24mn) | ||
| ziyourenxiang has quit IRC (Ping timeout: 248 seconds) | [09:21] | |
| ..................... (idle for 1h40mn) | ||
| erratic has quit IRC (Excess Flood) | [11:01] | |
| erratic has joined #arpnetworks | [11:08] | |
| ..... (idle for 23mn) | ||
| erratic has quit IRC (Excess Flood) | [11:31] | |
| qbit | so - for the dedicated boxes
i am hitting an issue with clocks on openbsd | [11:36] |
| brycec | Clocks as in... CPU? Or RTC? qbit
(Not that I have answers, I just help get details/triage) | [11:37] |
| qbit | RTC | [11:37] |
| *** | erratic has joined #arpnetworks | [11:38] |
| brycec | What's it doing? Drifting? Lagging? And to confirm, OpenBSD is the host OS on the dedi box? | [11:38] |
| qbit | basically this: https://marc.info/?l=openbsd-bugs&m=151430928212450&w=2
the preemption_timer i am wondering if it would be possible to get kvm-intel.preemption_timer=0 on what ever host has my dedi stuffs | [11:39] |
| brycec | What exactly is that? Some OpenBSD config() thing? A Linux string?
If you've got a dedi, you've got baremetal access, you can do whatever... | [11:40] |
| qbit | kernel boot option | [11:41] |
| brycec | Or have you been talking about a Thunder instance? | [11:41] |
| qbit | thunder
i call them dedi because : https://arpnetworks.com/dedicated "dedicated" | [11:41] |
| brycec | "dedicated resources" but yeah, it leads to confusion IMO
So to clarify, you're asking if you could arrange for the Thunder host machine could be rebooted with a kernel parameter? | [11:41] |
| qbit | guess i am asking about the posibility of doing that
i understand that likely there are more clients.. so if it isn't an option - i will have to figure something else out | [11:43] |
| brycec | Seems like a tall ask :p but that's my opinion. And like I said, I'm just trying to distill the problem/request to help everyone out. :) | [11:44] |
| mercutio | does this effect normal vps instances too?
it seems like quite a major openbsd bug | [11:44] |
| brycec | *affect :D | [11:44] |
| mercutio | yes :) | [11:44] |
| brycec | qbit: What was your `sysctl kern.timecounter` anyways? | [11:44] |
| qbit | seems to be anything in kvm
brycec: acpitimer0 | [11:44] |
| mercutio | i think the only major change to openbsd in the recent history has been the addition of virtio random device | [11:45] |
| brycec | So not exactly the same as the mailing list guy's | [11:45] |
| qbit | mercutio: negative - i am running -current
this is a grip of changes from 6.2 | [11:45] |
| mercutio | was it fine prior to -current | [11:46] |
| qbit | seems to have been fine prior to the meltdown stuff | [11:46] |
| mercutio | sigh
yeah so it probably will start happening on all virtual machine instances unless they fix it starting with the next release "FWIW, there are reports that this bug is absent from qemu-2.11.0. hmm i thought it was a kvm bug rather than qemu bug | [11:46] |
| qbit | https://marc.info/?l=openbsd-bugs&m=151439799628499&w=2
oh, you are probably already past that :P | [11:49] |
| mercutio | well it's mostly about kvm_intel.preemption_timer being disabled apparently helping
that made me think it was kvm issue uh, i don't think preemption_timer exists in the linux version being used | [11:51] |
| qbit | heh
it's a boot option (i put mine in grub) | [11:54] |
| mercutio | it's not in /sys/module
but it's in /sys/module on newer kernel | [11:54] |
| qbit | dang | [11:54] |
| mercutio | it's linux 4.4 | [11:54] |
| qbit | will /sys/module always reflect avaiable moduels? or maybe loaded?
ah | [11:54] |
| mercutio | not always | [11:54] |
| qbit | Not Always - Linux | [11:54] |
| mercutio | but if it's not supported in there you can definitely not change it without reboot | [11:55] |
| qbit | dang | [11:55] |
| mercutio | oh you can't change it while booted anyway | [11:55] |
| qbit | double dang | [11:56] |
| BryceBot | That's what she said!! | [11:56] |
| qbit | if you say so BryceBot | [11:56] |
| mercutio | https://www.spinics.net/lists/kvm/msg161757.html
this suggests one way that may help a specific instance qbit: this is quite easy to reproduce right? | [12:06] |
| qbit | step 1) install openbsd | [12:09] |
| mercutio | snapshot? | [12:10] |
| qbit | ya
https://ftp3.usa.openbsd.org/pub/OpenBSD/snapshots/amd64/ that's all - no more steps :D clock will drift even with ntp going | [12:10] |
| mercutio | hmm install62.fs is preinstalled? :)
i can't remember seeing that before | [12:11] |
| brycec | What do you mean "is preinstalled"? It's a dd'able version of the ISO, but still just the install media (bsd.rd) | [12:14] |
| qbit | it's bsd.rd | [12:14] |
| mercutio | oh
damn | [12:14] |
| qbit | with sets | [12:14] |
| brycec | qbit: Have you tested to see whether it can be reproduced from bsd.rd alone?
Or does it take the full bsd kernel config? (I know that bsd.rd omits a lot of ACPI stuff to keep its size down, among reasons) | [12:14] |
| qbit | i don't know if bsd.rd is enough to make it happen
yeah, i doubt it would work | [12:15] |
| brycec | Also, I would've pointed mercutio to http://mirrors.arpnetworks.com/OpenBSD/snapshots/ :P | [12:15] |
| qbit | s/work/be a good test case/ | [12:15] |
| BryceBot | <brycec> Also, I would've pointed mercutio to http://mirrors.arpnetbe a good test cases.com/OpenBSD/snapshots/ :P | [12:15] |
| qbit | :D | [12:15] |
| mercutio | BryceBot: that's where i'l pulling it from :)
oh oops :) | [12:15] |
| brycec | lol qbit | [12:16] |
| qbit | Feb 20 13:03:47 spark ntpd[5226]: adjusting local clock by 18.013269s
Feb 20 13:08:07 spark ntpd[5226]: adjusting local clock by 22.688790s | [12:17] |
| mercutio | damn | [12:18] |
| up_the_irons | I agree with mercutio , it seems like quite a major openbsd bug | [12:18] |
| brycec | up_the_irons: For reference, what version of Linux/kvm/qemu are Thunder machines running?
qbit: And just to be clear, *you* have only seen this behaviour with recent snapshots? Older snapshots were fine, as were older releases? | [12:21] |
| up_the_irons | 4.4.0-109-generic #132-Ubuntu SMP Tue Jan 9 19:52:39 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
ii qemu-kvm 1:2.5+dfsg-5ubuntu10.16 amd64 QEMU Full virtualization | [12:22] |
| qbit | brycec: if it was in previous snapshots - it wasn't enough ofa delay for me to notice
ha | [12:22] |
| brycec | I'm going to fire up a -current machine on a Proxmox machine, 4.4.62-1-pve + qemu 2.9.1-6~pve4 and see what happens... | [12:24] |
| qbit | i am not very observant
# date Tue Feb 20 13:21:04 MST 2018 # | [12:24] |
| mercutio | mine just finished installing | [12:24] |
| qbit | that's 6.2 | [12:24] |
| mercutio | i like how openbsd installs quite quick | [12:24] |
| qbit | Feb 20 13:19:22 db ntpd[2811]: adjusting local clock by 207.116095s | [12:25] |
| brycec | qbit: So you're saying 6.2 is also affected, cool (I have 6.2 VMs already in Proxmox :p) | [12:25] |
| qbit | ya | [12:25] |
| mercutio | hangon my vm is using acpihept0
will that change things? | [12:26] |
| *** | erratic has quit IRC (Excess Flood) | [12:27] |
| brycec | For the most part, my VMs stay just fine, but sometimes there are wild swings in drift, I'm guessing caused by load spikes in other VMs on the host. http://ix.io/N8h
By comparison, a 6.0 VM running on the same host has almost no time swing | [12:28] |
| mercutio | brycec: is it using acpihept0?
my time still seems right with -current heh maybe i need to trigger it with load? | [12:30] |
| brycec | Yes, both my 6.0 and 6.2 VM on the same Proxmox host are using acpihpet0. Only the 6.2 VM sees random drift. (Seeing it on Proxmox means it's not just an ARP issue :) And it's what I have to play with readily available.) | [12:32] |
| mercutio | (i'm testing on my desktop first)
my desktop has linux 4.15, and some recent qemu | [12:32] |
| qbit | yeah - this has been confirmed on vultr too | [12:32] |
| mercutio | i'm kind of curious if they've fixed it in linux or osmething
qemu 2.11.1 | [12:33] |
| brycec | 6.0 http://ix.io/N8s 6.2 http://ix.io/N8t Same exact kern.timecounter values. | [12:34] |
| mercutio | ahh cool
that's curious,y ou got a lot more adjustments sicne the 13th | [12:34] |
| brycec | "Something happened" on the 13th it seems. | [12:36] |
| mercutio | to both of them | [12:36] |
| brycec | And re-evaluating my output, I guess 6.0 also saw some lagging afterall. | [12:36] |
| mercutio | this doesn't seem extreme though
what qbit was talking about seemed more extreme | [12:37] |
| brycec | I disagree, 10s+ adjustments are extreme :p | [12:38] |
| mercutio | i mean it's not good
but wasn't there like minutes of drift like a lot bigger difference | [12:38] |
| brycec | But these 2 particular VMs of mine are real small stuff, not doing any major load (they're effectively bastion hosts). I suspect with heavier load, it exacerbates the issue. | [12:38] |
| mercutio | yeah it may | [12:39] |
| qbit | it's def worse on machines with high load | [12:39] |
| mercutio | i need to compile something :) | [12:39] |
| brycec | So I guess I'm back to: a) Doesn't seem to matter which OpenBSD release, even 6.0 is affected to a degree, albeit perhaps lesser. b) Even newer kernel+kvm (than ARP, but not "newest") are affected. | [12:39] |
| anisfarhana | You guys are crazy. | [12:41] |
| qbit | likely the meltdown stuff in current snaps is gonna make it worse too | [12:41] |
| *** | erratic has joined #arpnetworks | [12:41] |
| mercutio | i just realised something i'm not testing with smp
is this only affecting smp hosts, or in general? s/hosts/guests/ | [12:42] |
| BryceBot | <mercutio> is this only affecting smp guests, or in general? | [12:43] |
| qbit | mine are all single cpu | [12:43] |
| mercutio | ah ok | [12:43] |
| brycec | Ditto for me (I'm pretty sure) | [12:44] |
| mercutio | i thought was on thunder | [12:44] |
| qbit | it is - but mine is proxmox -> openbsd vms | [12:45] |
| mercutio | ah ok
well i did a kernel compile, and it failed to compile... maybe -current has broken compiling 6.2 time is still right so yeh i think either newer kernel or newer kvm has fixed it | [12:45] |
| *** | erratic has quit IRC (Excess Flood) | [12:50] |
| qbit | huh
well that's cool it seems to sync back up once load has gone down | [12:56] |
| *** | erratic has joined #arpnetworks | [12:58] |
| ................. (idle for 1h24mn) | ||
| erratic has quit IRC (Excess Flood) | [14:22] | |
| erratic has joined #arpnetworks | [14:28] | |
| .... (idle for 19mn) | ||
| mercutio | qbit: what kernel is promox running under?
i'm thinking that the underlying thunder thing may make less difference than the proxmox layer.. | [14:47] |
| .... (idle for 15mn) | ||
| qbit | mercutio: 4.4.98-6-pve | [15:02] |
| mercutio | ahh so similar to ubuntu kernel
oh maybe proxmox doesn't haev a recent kernel available to try The current stable 4.x release uses latest Ubuntu based kernel, which will be regularly updated. The first stable 4.0 release is based on 4.2 Linux kernel. it's actually the same kernel even :) | [15:02] |
| .............................. (idle for 2h27mn) | ||
| qbit | heh | [17:31] |
| .................................................... (idle for 4h18mn) | ||
| *** | erratic has quit IRC (Ping timeout: 260 seconds) | [21:49] |
| ...................... (idle for 1h47mn) | ||
| erratic has joined #arpnetworks | [23:36] | |
| ..... (idle for 20mn) | ||
| fIorz_ has joined #arpnetworks
fIorz has quit IRC (Remote host closed the connection) | [23:56] | |
| ↑back Search ←Prev date Next date→ Show only urls | (Click on time to select a line by its url) |