***: dj_goku_ has quit IRC (Ping timeout: 250 seconds)
ziyourenxiang has joined #arpnetworks pyvpx: everyone adding NTP to their dhcpd after the google announcement? :p mkb: what google announcement
please don't say it's the fake time stamp thing
leap second I mean ***: Lucifer333 has joined #arpnetworks plett: pyvpx: The ones that smear leap seconds? No, it's a silly idea and client applications need to cope with their clocks changing under them for other reasons, so dealing with a leap second should be no problem ***: ziyourenxiang has quit IRC (Quit: Leaving) mercutio: things neding to doesn't describe reality at all jcv: i've written more than enough code that cares about leap seconds (satellite data) to think google's smear idea is terrible mercutio: is it terrible for good code, or terrible for bad code? brycec: mercutio: zeit.arp is also seeing the traffic increase. It's setting off my monitoring constantly :(
But thanks for mentioning it and the nanog list, at least I can follow along...
(I really wondered. At a glance, the traffic seemed legit so I put off investigating it.) mercutio: it's mostly mobile providers seemed to be the thing that was known so far brycec: I don't have finer-grained record on it, alas, so I can't break down connections by source or what their queries looked like. But I do monitor overall bandwidth which has tripled. 12 Dec was averaging 550kbps, the next few days are 723kbps, 1.26mbps, and 1.45mbps mercutio: ahh
that's much lower than some people see
some people have seen 20 megabit brycec: Yeah I saw Dan Brown's stats (http://seclists.org/nanog/2016/Dec/161) but still that's ~10% of his total traffic, that doesn't seem disproportionately high.
Well zeit.arp is a relatively low stratum, 2 or 3 I think.
(I assume higher stratum, eg 1, get more traffic from the NTP Pool)
In any case, it's better to talk about relative increases rather than absolute numbers. He's see quadruple the bw, we've seen triple, it's all... weird. mercutio: hmm brycec: (Zeit used to do ~5GB/day, yesterday it did 15GB :/) mercutio: yeh
some of the servers in thsi neck of the woods bailed ouit recently
which i think pushed traffic up further
and they're seeing US traffic as well for some reason
i dunno why there's so many posts here
err on nznog brycec: And this is the traffic from my own VPS which I also have setup in the Pool under a fairly low stratum) http://imgur.com/a/Ixs1U mercutio: there's 19 posts on nznog, 13 posts on nanog
hmm it looks like it started to go up slower
on monday brycec: Looking at the current traffic to Zeit (and hammering rDNS) I'm seeing a surprising number out UK clients
(BT, Ireland ISPs, etc)
(Virgin Mobile, Telus Canada which I know is not UK)
(Norway, Sweden, Germany...)
I think I'm most surprised to see requests from AWS EC2 instances though.
Not a ton, but a few.
(France, The Netherlands, Switzerland, Brazil)
(Belgium, Argentina) mercutio: hmm that is curious
so yeah there's two things, the geo location seems wrong
and no-one knows why there's heaps more traffic brycec: I was hoping it was something obvious like "Amazon turned on NTP inside all new EC2 instances" but... 1) That's dumb, and 2) I should see more traffic then, probably. ***: Lucifer333 has quit IRC (Quit: Leaving)
hive-mind has quit IRC (Remote host closed the connection)
hive-mind has joined #arpnetworks
dj_goku has quit IRC (Remote host closed the connection)
ziyourenxiang has joined #arpnetworks
dj_goku has joined #arpnetworks
Nahual has joined #arpnetworks nathani: what tool / software / config can I use to authenticate BGP prefixes to originate from respective ASNs? mercutio: RPKI
it's hardly used though nathani: like DNSSEC :-)
https://www.youtube.com/watch?v=P65XdTlk4vA BryceBot: YouTube video: "Jonathan Zittrain: The Web as random acts of kindness" by TED mercutio: DNSSEC is used heaps now
dnscurve is hardly used ***: Nahual has left nathani: @google dnscurve BryceBot: 1,440 total results returned for 'dnscurve', here's 3
DNSCurve - Wikipedia (https://en.wikipedia.org/wiki/DNSCurve) DNSCurve is a proposed new secure protocol for the Domain Name System ( DNS), designed by Daniel J. Bernstein. Contents. [hide]. 1 Description; 2 Security  ...
GitHub - mdempsky/dnscurve: Tools for DNS curve implementation (https://github.com/mdempsky/dnscurve) Tools for DNS curve implementation. Contribute to dnscurve development by creating an account on GitHub.
DNSCurve – Wikipedia (https://de.wikipedia.org/wiki/DNSCurve) DNSCurve ist eine Technik zur sicheren Auflösung von Domain-Namen in IP- Adressen. Autor des im August 2008 veröffentlichen Protokoll-Vorschlags ist der  ... nathani: cloudflare has support for dnssec
even on their free tier -: brycec makes use of it :) nathani: I used to be a heavy dnsmadeeasy user, but can't beat free and all sorts of caching / security features with cloudflare
does pool.ntp.org resolve to zeit for close clients?
ie is it part of the public pool? brycec: It should, yes.
(otherwise I have no idea how France, Belgium, Germany, the Netherlands, UK... got the address) nathani: was the spike in traffic across both v4 and v6? brycec: I can't say for certain, I only monitor the traffic at the interface level.
Doing periodic tcpdumps, traffic is 99% v4 nathani: do you also monitor skew and time corrections on the vm itself? brycec: What's really interesting to me is that, at least according to tcpdump's protocol identification, ip6 traffic is exclusively ntpv2 and ntpv3 clients, while ip traffic is about 98% ntpv4
Yes. It's pretty stable. And perks of NTP, it handles itself fairly well. nathani: you would think the ipv6 clients would be more capable and request later version of ntp protocol
dns queries to pool.ntp.org dont seem to return AAAA records when requested brycec: But 2.pool.ntp.org does
So any clients with [0123].pool.ntp.org configured, as I've often seen in default ntp.conf will still hit it nathani: the nanog list mentioned an IOT provider that had configured something differently
didnt name the provider or device though brycec: Did it? I don't remember seeing anything like that http://seclists.org/nanog/2016/Dec/index.html#159 nathani: nznog actually
I have them in the same label in gmail
"The chatter in #ntp on IRC infers that it was through a change made by a IoT vendor (though that's all the info that's been given, so take that with as much salt as you wish)."
https://list.waikato.ac.nz/pipermail/nznog/2016-December/022411.html brycec: Oh nznog :p nathani: Folks down under need ntp too :-) brycec: (Thanks for thelink)
(I didn't have a link to nznog archives) nathani: there is also AUSnog which I follow: http://lists.ausnog.net/pipermail/ausnog/ mercutio: yeh i dunno why nznog had so much discussion :) nathani: time servers are 'critical' infrastructure for the internet, kinda like dns servers 'maybe' - it is essential to have them up and running and a spike in traffic of such extent can lead to insufficient capacity to deal with legitimate queries assuming the excess traffic is not legit up_the_irons: so what's all this about increase in NTP traffic....
why exactly would Zeit be getting more traffic now? brycec: up_the_irons: because zeit is a member of pool.ntp.org
And pool.ntp.org is seeing an unexplained increase in traffic
It's legitimate traffic so far as anyone can tell, at least. (And not something nefarious like a DDoS or amplification attack) nathani: https://lists.ntp.org/pipermail/pool/2016-December/007997.html
this guy had to shut off his ntp server to get his firewall working brycec: (Though to be fair, it was a Cisco ASA *rimshot*) nathani: Just a thought, yesterday was Microsoft patch day. If MS added the pool to all the Windows clients out there, that could certainly account for this traffic.
^ lol brycec: (yeah saw that message)
Windows still defaults to time.windows.com last I checked. nathani: where is the page that shows you health of individual servers etc
I think I was looking for http://www.pool.ntp.org/scores/208.79.89.249 brycec: http://www.pool.ntp.org/scores/2607:f2f8:a650::3
Yeah
Zeit ip6 http://www.pool.ntp.org/scores/2607:f2f8:0:102::2317
Zeit ip4 http://www.pool.ntp.org/scores/208.79.89.249
for those interested up_the_irons: brycec: ah OK
I forgot it was part of that pool brycec: lol
up_the_irons: Sorry about the unexpected, unexplained tripling in traffic nathani: http://irclogger.arpnetworks.com/irclogger_log/arpnetworks?date=2014-06-19,Thu&sel=389#l385
wow its been 2.5 years brycec: (Geez look at nathani pulling a brycec, quoting the logs) nathani: too bad brycebot didnt pull the quote from the url and paste it into the channel :-) brycec: Maybe someday up_the_irons: brycec: do you know what the Mbps is? brycec: up_the_irons: Yes.
Today's average is 1.89mbps
Yesterday's is 1.45mbps
day before 1.26 nathani: thats like 20gb/day up_the_irons: OK tnx
so not bad nathani: I guess folks are concerned if it keeps increasing like that
the list mentioned 20mbps in some cases
also its small packets so max pps on firewalls etc brycec: 16.72GB so far today, yes nathani
14.92GB yesterday
fwiw zeit is configured as 100mbps North America mercutio: wow
i suppose there's lots of higher bandwidth ones
it sounded like 50 megabit ones were getting hit hard before brycec: (today's average is up to 1.91mbps, total 17.01GB, 8.58GB inbound + 8.42GB outbound) nathani: how is cpu load?
have you seen https://developers.google.com/time/ brycec: fairly low, 0-10% CPU usage
I saw mentions of it. I... don't approve.
(of "smearing") nathani: what about all the apps that cant handle leap seconds brycec: Fix the app.
Duh :p
Frankly I don't think I've encountered an application that can't handle leap seconds nathani: if folks use standard ntp you can correlate events from different systems and be sure the timestamps refer to the same time. No translation as in the case of smearing brycec: (I mean, I'm not saying affected applications don't exist. I just haven't encountered one personally)