mercutio: i assume so cos someone asked if it was over ipmi :) brycec: https://twitter.com/arpnetworks/status/476982306163286017 was 22 minutes ago, so presumably the upgrade has bee going for the last 17 BryceBot: TWITTER: Maintenance window on host kvr15 will begin in about 5 minutes (Thu Jun 12 07:00:18 +0000 2014) mnathani: Broadcast message from root (Thu Jun 12 03:05:31 2014):
The system is going down for system halt NOW! SpaceDump: :D CaZe: Because it doesn't make business sense.
They do have 24 hour drive throughs where it does make sense. brycec: When I was at uni, there was a Starbucks open 'til 11pm
(the library was open 24hr, and the city scene were very active late into the night) ***: jm|laptop has joined #arpnetworks jm|laptop: did it pop?
there it is! plett: jm|laptop: Oh. Hello :) jm|laptop: Oh. Hi. :)
https://twitter.com/arpnetworks/status/476982306163286017 was it that? BryceBot: TWITTER: Maintenance window on host kvr15 will begin in about 5 minutes (Thu Jun 12 07:00:18 +0000 2014) jm|laptop: Thu 12 Jun 09:08:07 BST 2014 mercutio: oh it seems backup but going slow
well taking ages to login up_the_irons: cuz all VMs doing fsck mercutio: hmm fresh login 2.89 load average
even with acpi shutdown?
oh uptime is only 5 minutes
ok :)
i thought it took about half an hour
so yeah, the fsck hasn't been done in x days thing SpaceDump: weee
I had done something wrong on that vps. Everything came up again after startup. :D mercutio: hmm i need to change to virtio now
well don't need but should jbergstroem: are there any known issues with network connectivity/routing? One of my CDN pops in Sydney doesn't seem able to reach my server
Can anyone else try to do a route from arp to 23.235.41.22? ***: z310 has quit IRC (*.net *.split)
jcv has quit IRC (Quit: leaving)
jcv has joined #arpnetworks plett: jbergstroem: http://pastebin.com/fjx9GiWC jbergstroem: plett: a while ago, seems to have fixed itself now: https://gist.github.com/jbergstroem/fd23d8ca83c9462ec807 BryceBot: Gist: "https://gist.github.com/fd23d8ca83c9462ec807" plett: Cool. Glad it's working again then jbergstroem: just after i rerouted shit at my cdn's place :/ ***: z310 has joined #arpnetworks brycec: Came across this long-running mtr that spans the Level3 turn-up, thought it might be interesting to some https://dl.dropboxusercontent.com/u/3167967/screenshot_2014-06-12_09-58-07.png ***: mus1cb0x has joined #arpnetworks mus1cb0x: since 14 was upgraded this week, i've been getting noticably increased 'broken pipe' to it
i know it isn't my inet link because i also have a ssh session to a different arp server and it remains up
before this, my connection to 14 was rock solid. anyone else on 14 notice degradation in QoS? m0unds: mus1cb0x: which OS? mus1cb0x: freebsd 9.2 m0unds: did you enable virtio? mus1cb0x: where would that have been done?
in rc.conf or something?
i'm running stock -release kernel/base m0unds: mus1cb0x: have you made any changes to configuration since the maintenance event? mus1cb0x: none m0unds: huh, weird
see anything in logs? mus1cb0x: up_the_irons: any other reports of this?
which logs would you suggest i look at? m0unds: you could just try grep-ing the logs folder for your interface name
maybe look for watchdog as another item mus1cb0x: do you know how i could grep all files in the current dir for str ?
em0 is the if brycec: grep -rn em0 .
(the key here is -r) mus1cb0x: ty brycec
http://pastebin.com/85d7sCwe is everything that matches em0 in /var/log
i don't know what "52:54:00:27:22:11" is, but it appears before the maintenance window so i doubt it's an issue
seems like there's a lot of promiscuous mode being enabled then disabled, not sure if that could be cleaned up
but again, appeared before maint. window also brycec: mus1cb0x: You mean, em0: Ethernet address: 52:54:00:27:22:11
It's the MAC
As for "promiscuous mode" you were most likely running tcpdump ***: httpduck1 has quit IRC (*.net *.split)
httpducks has joined #arpnetworks acf_: brycec: that's really cool
the mtr
I see three paths?
ntt, level3, and any2 mus1cb0x: ah mercutio: paste has been ermoved
i see none :) acf_: https://dl.dropboxusercontent.com/u/3167967/screenshot_2014-06-12_09-58-07.png
mercutio ^ ***: tabthorpe has quit IRC (Ping timeout: 252 seconds)
tabthorpe has joined #arpnetworks
avj has quit IRC (Ping timeout: 264 seconds)
raptelan has quit IRC (Ping timeout: 264 seconds)
wallshot1 has joined #arpnetworks
CaZe has quit IRC (Ping timeout: 252 seconds)
mnathani has quit IRC (Ping timeout: 252 seconds)
jcv has quit IRC (Ping timeout: 264 seconds)
brycec has quit IRC (Ping timeout: 264 seconds)
acf_ has quit IRC (Ping timeout: 264 seconds)
eryc has quit IRC (Ping timeout: 264 seconds) wallshot1: i'm guessing all those timeouts confirm what i was wondering ***: jlgaddis has joined #arpnetworks wallshot1: network or vps host outage perhaps? i should try to vnc in and see what it says from the inside jlgaddis: internet b0rked? dangel: wallshot1: Agreed. Me too. jlgaddis: yeah i had two vps' drop in the last three minutes ***: raptelan has joined #arpnetworks
acf_ has joined #arpnetworks
jcv has joined #arpnetworks
CaZe has joined #arpnetworks KILLALLHUMANS01: I'm still connected to my VPS just fine
And some of those VPS->IRC connections are still good, but not VPS->Freenode(whatever server I'm on) m0unds: still connected to mine too dangel: I can't ssh into my node. wallshot1: ssh timeouts turned into ssh slow-but-successful KILLALLHUMANS01: I'm seeing a bunch of OpenVPN connections flap thoguh m0unds: hm acf_: hmm that was strange dangel: Oh, there it goes. KILLALLHUMANS01: So I'm thinking there's aroute having issues. wallshot1: and pidgin is reconnected to my jabber server again, yay jlgaddis: and everything's back up lol ***: eryc has joined #arpnetworks
eryc has quit IRC (Changing host)
eryc has joined #arpnetworks m0unds: weird. didn't notice anything here at all. acf_: mtr -4 asimov.freenode.net KILLALLHUMANS01: │13:51:52 *status | Disconnected from IRC (Connection reset by peer). Reconnecting... ***: mnathani has joined #arpnetworks mercutio: oh it means the route changed acf_: the two hops after s7.lax.arpnetworks.com are ??? mercutio: it may have b een withdrawn and readvertised KILLALLHUMANS01: The route can change without destroying the connection... Unless the route changed to something unroutable. mercutio: well it's strange for 10.10.10.6 to appear in it acf_: what are you tracing to mercutio? mercutio: acf: nothing, i'm lokoing at your mtr :) acf_: oh I don't see 10.10.10.6... mercutio: yes you do acf_: it goes via Level3 mercutio: hop 3 second number
err 2nd line acf_: http://paste.unixcube.org/k/7b6ab9 mercutio: the screenshot
that kinud of thing doesn't show up in short mtr's normally acf_: oh that mercutio: only when you leave mtr running in the backgruond
and catch a route dropping
or changing path acf_: yes, that's brycec's trace from the Level3 switch over
it's cool mercutio: but when hop 10 says los angeles, it suggests it probably dropped
and looped a little KILLALLHUMANS01: Yeah that's an old, long-running mtr ***: brycec has joined #arpnetworks KILLALLHUMANS01: Oh look at me, I'm back mercutio: oh youre bryce m0unds: yeah
haha KILLALLHUMANS01: am brycec: am mercutio: i was wondering who it was acf_: same here lol mercutio: i'm sure i'd seen the name before KILLALLHUMANS01: This is a connection I leave up from another host, so... lurking mercutio: heh
freenode has mostly been stable for me recently
[mercutio] idle: 00 hours 00 minutes 03 seconds, signon at: Sat, 03 May 2014 06:31:08 m0unds: 14:57:02 freenode -- | [m0unds] idle: 00 hours 01 minute 25 seconds, signon at: Mon, 21 Apr 2014 17:26:55 mercutio: there's been a netsplit in that time
it used to be pretty unstable ***: mus1cb0x has left "WeeChat 0.4.2" mercutio: the vast majority of people in this channel seem to idle m0unds: yep -: dangel pets mosh. m0unds: i guess apd cornered an armed robbery suspect in our employee parking lot
haha RandalSchwartz: apd?
[something] police department, I would guess m0unds: yep
albuquerque RandalSchwartz: well - that explains it :) mercutio: hmm i left a mtr to amazon running
after it goes into amazon's network it hops around a lot wallshot1: i'd expect that at any good sized org as clusters are put into and out of rotation for maintenance
with entire data centers going in and out of rotation at any given time brycec: Even for simple load balancing mercutio: wall: sounds complicated
hop 14 has 8 differnet ip's
but they all start with 205.251
it's 21 hops
and 11 hops to amazon
so 10 hops within their network
i probably should stop this mtr
id on't know where it's too :/ wallshot1: that is indeed a few hops in their network before hitting the endpoint mercutio: ec2-54-214-15-46.us-west-2.compute.amazonaws.com
someone gave me that host to try us west?
http://54.214.15.46/videos/Yelle%20-%20Comme%20Un%20Enfant%20%28Freaks%20Remix%29.webm
what's that heh acf_: watch it
the whole thing mercutio: ahh so it you acf? acf_: yep mercutio: why are you on ec2 acf_: it has super good transit mercutio: i don't know if i have a webm player acf_: https://www.youtube.com/watch?v=hAiu04sUDMg BryceBot: YouTube Music: "Yelle - Comme Un Enfant (Freaks Remix)" by Nathan Barnatt (5m 41s), 141,973 views, 4,515 likes and 29 dislikes. Uploaded 2014-01-03T21:48:54.000Z. mercutio: i dunno i have 2.7% packet loss to it :/
and 0.2% to paix amazon
arp never has that high loss :/ acf_: hmm, I don't see that m0unds: neither do i acf_: what do you do when the mtr goes off the terminal?
too long mercutio: http://pastebin.com/epc89FBq
i resize the window
i did alt enter at first acf_: my window won't go any bigger lol mercutio: then i realised it was way too wide to cut and paste
i'm using 1440p acf_: ah, ok brycec: 50pkts no loss mercutio: and it's over half my vertical screen size
so at 720p i'm sure you'd struggle.
i have 58,000 packets
:) ***: wallshot1 has quit IRC (Read error: No route to host) acf_: No route to host? how does that happen? m0unds: depends ***: wallshot1 has joined #arpnetworks acf_: Level3 seems kind of jittery since it dropped out earlier today up_the_irons: jbergstroem: plett : yeah i got some alerts of peering trouble earlier today
m0unds: whenever you mention apd it always reminds me of breaking bad, lol m0unds: up_the_irons: bahaha brycec: And it probably will for years to come
One of these days I'm going to finish watching BB m0unds: yeah, it's either BB or USDOJ investigation
alternates between those two things, since all people know about ABQ is that breaking bad was shot here and that APD is under investigation for use of force brycec: I was unaware of the latter, actually
So congrats on that m0unds: haha, i'm surprised. there was a shooting event where APD killed a batshit insane homeless guy and it made national news
and stupid anonymous targeted cabq.gov and apd's stuff up_the_irons: i was unaware of the latter too m0unds: well geez, i'm disappointed now
haha
lots of shootings by apd in the last 10 yrs brycec: I've heard of DOJ investigating similar charges for LAPD and SPD (Seattle), but ABQ slipped under my very, very high radar. m0unds: haha, SPD was turned into a metro PD under recommendation by usdoj brycec: o.o m0unds: or am i thinking of LVPD
or something brycec: Ok. I have no idea what that "means m0unds: federally administered police dept
something like that brycec: oh my
The town I live in has 6 cops :)
The PD is also administered (or governed, or something like that) by the county sheriff m0unds: yeah, lots of small agencies work that way
actually, seems like LV metro pd is that way too
and they're huge (officer count-wise) jbergstroem: up_the_irons: ok, thanks for letting me know up_the_irons: it's the jurisdiction radius, basically. counties are protected by sheriff's departments. within a county, the cities can have their own municipal police department (and many do, e.g. LAPD, BPD (Burbank), GPD (Glendale), etc... around my area)
cities that don't have their own PD get automatically covered by the sheriff
and lastly, some cities simply "outsource" their PD to the sheriff (e.g. West Hollywood)
those are called "contracted" cities
and is why you see sheriff cars in an otherwise LAPD controlled area
but i still don't know what "federally administered police dept" would be
either jbergstroem: up_the_irons: btw, next time i have similar issues should i just hit support@ ? up_the_irons: jbergstroem: yup jbergstroem: up_the_irons: done m0unds: up_the_irons: usdoj handles administrative duty and training stuff for a police agency up_the_irons: :)
m0unds: roger ***: siignal has joined #arpnetworks m0unds: haha, add qwest/centurylink to the list of ISPs who run their NTT peering way too hot
ugh ***: siignal has left acf_: Verizon, AT&T, Comcast, CenturyLink/Qwest, TWC, Cox, Charter m0unds: centurylink announces bogons
yay acf_: so does Level3
according to bgp.he.net
and Verizon
even the Verizon <-> Comcast peering is hot m0unds: it's for the best - i don't want my house to burn down mercutio: acf: i kind of wish there was more of an international monitoring/graphing/record of this stuff
broken down as much as possible acf_: yeah. it's kind of surprising that one doesn't exist actually
considering the magnitude of some of these problems mercutio: probably because it's hard
i mean you need 10,000+ peopel testing to really get good results. acf_: I wonder if nanog ring could help any
you have access to test points from a bunch of different nsps/etc... mercutio: still isn't it like 20 locations?
you kind of want more ilke 50 locatiosn
oh it' smore complicated than that
like if you can select different providers in 50 locations.. gizmoguy: https://ring.nlnog.net/ m0unds: eggnog ring acf_: https://ring.nlnog.net/participants/ mercutio: i got 50 from 7 countries and 6 places in each country + a bit
i don't relaly know good numbers
but i imagine some of them are near each other gizmoguy: nlnog ring is probably the best combined network monitoring project outside of perfsonar
our software used to be used to monitor the nlnog ring, not sure if it still is mercutio: so like in the US, you want to test at least dallas, los angeles, san jose, virginia, new york, chicago, phoenix, denver, miami, seattle, ..
i'm missing a few important network locations right? m0unds: i still like ripe atlas, despite the black box nature of it mercutio: and test the different providers in each of those regions acf_: so nlnog is just like a big shell access exchange? gizmoguy: https://stats.es.net/perfSONAR/directorySearch.html mercutio: can you do throughput tesitng on nlonog?
nlnog ring gizmoguy: I suspect so mercutio: i imagine it's mostly about network health gizmoguy: what we did for them acf_: oh cool
https://ring.nlnog.net/toolbox/ gizmoguy: is create a full mesh of all participants, and do regular traceroute/icmp mercutio: some people have a few, but some are jst one gizmoguy: in the mesh mercutio: and record networks traversed and jitter and so on? gizmoguy: naw they turned off our software
https://ring.nlnog.net/news/2012/10/root-cause-analysis-using-amp/ mercutio: damn gizmoguy: here's an old blog post mercutio: is it open source? gizmoguy: not at the moment
http://wand.net.nz/amp/
we're in the middle of a 4 year project to rewrite the whole thing mercutio: i think the best way to do testing is to involve enough people that local issues on less well connected sitse don't matter
rather than having "well connected" test sites.
gizmoguy: do you ever look at geekzone forums? they were talking about congesting happening on 2.4 gigabit ONT's that go to 24 houses or such.
but a lot of people don't seem to really understand congestion well.
and likely places rather than unlikely places for it to happen gizmoguy: I try and stay well away from that place mercutio: haha good idea
i still think congestino is not widely understood gizmoguy: do you have a link to that forum post?
I have a few people I would like to share that with :) mercutio: it's on the 1 gigabit fibre thread acf_: like the 24 houses are using up 2.4 gigabit of bandwidth? mercutio: http://www.geekzone.co.nz/forums.asp?forumid=49&topicid=147171&page_no=6#1064686
somewhere in there
acf: gpon is 2.4/1.2 gigabit
2.4 down 1.2 up
adn gpon supports oversubscription
so you can have 24 customers with gigabit connections.
the question is, is it likely to congest?
google fibre is the same diff
shoudl be calling it "google fiber" because it's only available in the US?
it's also probably less than 24 people on a node in practice m0unds: the iprovo thing is such a boondoggle mercutio: iprovo? m0unds: paid for by taxpayer money and the city sold it to google for $1 mercutio: oh
ir ead about it m0unds: https://transmission.xmission.com/2013/04/18/the-1-fiber-optic-network mercutio: but google are paying $30 million on construction loans m0unds: which is small potatoes mercutio: i think it sonuded reasonable enough to me
did you see the thing about them paying $5/month in taxes wehterhet or not they take it up?
i can't tyo. s/wehterhet/whether/ m0unds: hahaha mercutio: i can't type my mistypings either can it.
-t
argh gizmoguy: urgh myfullflavour m0unds: gah, i'm dropping keystrokes like crazy mercutio: gizmoguy: they're not around here, so dont' know much about them. gizmoguy: I designed and built their first network
they didn't pay me mercutio: i suppose you can see i posted a bit heh
eek gizmoguy: well they paid 1% of the bill or something mercutio: well apparently they're upgrading their core network now
lots ofp eople are upgrading core networks atm]
but yeah, i was trying to determine if any sites could push anythhing close to gigabit
the fastest speeds i've had are to you ai think gizmo :) gizmoguy: :) mercutio: but i imagine that server has less load than citylink's ftp mirror etc
and it's rare to have much nz content of any good size gizmoguy: http://ps02.reannz.co.nz/iso/
see how fast you can pull that down at mercutio: it's not loading acf_: same mercutio: it's reannz only isn't it? acf_: so the question is
when everyone has google fibre
then what happens to our NTT congestion? gizmoguy: mercutio: oh yes right. sorry those machines aren't on the internet vrf mercutio: google use ntt i think? acf_: hmm idk
its any2 from arp mercutio: acf: it's ape from me (nz peering exchange)
it's good they're well peered at least acf_: they're peered directly to amazon too
and HE mercutio: via telecom?
err global gateway acf_: ? mercutio: what ip
is it like 210.55 or something acf_: http://paste.unixcube.org/k/2cad50
top HE, bottom amazon
not sure what you're looking for mercutio: that's google
isnt' it? acf_: teah
yeah mercutio: oh right
i thought you meant gizmo's server for some reason
sorry i'm slow
but that's google's web site
their fibre is different
i tracked down an ip ages ago acf_: right mercutio: but i lost it
they don't have peering in kansas
at least not well peered
and their traffic goes via dallas/chicago acf_: hmm I can see why that would be bad
can you buy transit from google fiber? mercutio: well kansas sucks for peering in general
it's one of the annoying places :/
comcast and verizon suck for peering there too. acf_: yeah, I haven't much experience with kansas
http://bgp.he.net/AS16591 mercutio: so what it looked like google were doing, is doing their own backhaul to places with peering
then buyign bandwidth adn peering there acf_: that would make sense mercutio: i've got a friend in wichita
which is close enough acf_: what kind of backhaul do they have? mercutio: he can download at 3 megabytes/sec from nz at least. acf_: if everybody has gigabit... mercutio: on comcast i think that was m0unds: they don't all have gbit
they have svc levels like anything else mercutio: acf: they've probably got 10 gigabit links
which they can upgrade to 40 gigabit when they congest.
you have to have a lot of users to congest dual 10 gigabit linsk. acf_: I guess
so they have 2 plans mercutio: and it's more prudent to get links to more places than to make the pipes fatter. m0unds: do you have a file to grab from something in nz?
i'm curious mercutio: m0unds: there's heaps of various files around :/ acf_: mercutio: ok I can see that mercutio: http://202.49.71.24:24/10m
that's a 10m test file on my server acf_: or potentially http://ftp.nz.debian.org/debian/ mercutio: my rationality is that once files get big you're not waiting for them to download
so it's better to test with files that aren't huge m0unds: my experience with even domestic linux mirrors is that they're under pretty regular load and can vary wildly gizmoguy: 0.9s download time on that mercutio m0unds: ~500KB/sec mercutio: gizmoguy: that seems slow
oh hangon 0.9 isn't too bad
that's 10mb/sec
are you connected at 100 megabit? gizmoguy: 11.0M/s in 0.9s
yup 100mbit mercutio: beacuse that's basically 100 megabit m0unds: i can grab it faster from home than my arp vm
hah
a mercutio: m0unds: haha
it's ntt to arp
i dunno if that says anything :/ m0unds: outbound is any2
comcast it's level3 mercutio: yeah
comcast forward path is verizon i think m0unds: level3 for me mercutio: yeah verizon m0unds: or fwd from you? mercutio: forward from me m0unds: 195ms mercutio: 500kb/ssec is actually painfulyl slow :/
i wonder why it's going so slow acf_: verizon/comcast peering seems a bit congested atm
through lax m0unds: ^ could be that, because it routes me through lax mercutio: yeah it will route via la or sj
to nz m0unds: the only time i see sjc for stuff out west is when things are screwy acf_: http://kremvax.acfsys.net/smokeping.cgi?displaymode=n;start=2014-06-12%2017:04;end=now;target=Remote.verizonnet~comcastep m0unds: because it typically does abq > den > sjc > elsewhere
or sometimes, abq > den > dfw > sjc > elsewhere mercutio: but yeah, nz is pretty good for doing 100 megabit speeds
but i really doubt that much is connected at any faster than 1 gigabit
asnd like i'm pretty sure gizmo's being slowed down by a bsaically arbitary 100 megabit limit
and that if he cared enough he coudl get it fixed
to gigabit
like one router or switch or such m0unds: haha, those debian mirrors are using a cdn mercutio: nz.archive.ubuntu.com does which is what i use for testing nz speeds normally
but can't give you :(
cos it is unfair
cdns do complicate things m0unds: yeah, the node it's giving me for the debian one is in san jose
that one is the same
actually
haha
16. ubuntu.citylink.co.nz 0.0% 13 43.5 43.4 41.5 44.6 0.8 mercutio: ftp://wits.cs.waikato.ac.nz/ispdsl/2/20100106-030946-0.dsl.erf.gz
i found something on gizmo's server to link :/
but it's rated limited at 10 megabit on ipv4, and needs ipv6
to go faster m0unds: and i can't do v6 because my v6 tunnel is to arp, which is congested atm mercutio: the pseed isn't great from arp for that too m0unds: 58.5KB/s eta 2h 50m mercutio: 551k/sec average at 9.6mb m0unds: ^ that's via the tunnel mercutio: i tried to ^C around 10mb
hahaha wow m0unds: 604KB/s eta 11m 55s that was direct
via v4 mercutio: weird gizmoguy: i can give you a big file on non-ratelimited connection if you want mercutio: gizmo: how about a 10mb file? :) m0unds: for science gizmoguy: http://wand.net.nz/boot/mirror/isos/CentOS-6.4-x86_64-LiveDVD.iso
oh
10 M
sure mercutio: you can just download my 10mb file
it's /dev/urandom though
dd if=/dev/urandom of=10m bs=1024k count=10 m0unds: ] 16,915,140 3.70MB/s
it's bouncing around, but highest speed was 5.7MB/sec mercutio: there were some dodgy vps providers serving /dev/zero content over https gizmoguy: http://wand.net.nz/boot/mirror/isos/10m mercutio: https compresses such files and makes it faster
100 10.0M 100 10.0M 0 0 71.4M 0 --:--:-- --:--:-- --:--:-- 71.9M gizmoguy: waikato netwokring is very broken at the moment
unfortunately mercutio: oh it was hitting proxy oosp
100 10.0M 100 10.0M 0 0 14.9M 0 --:--:-- --:--:-- --:--:-- 14.9M m0unds: i hit 2.77MB/sec on that one mercutio: curl -v http://wand.net.nz/boot/mirror/isos/10m > /dev/null 0.01s user 0.03s system 5% cpu 0.748 total
hmm so it's not really a lot faster than your 100 megabit connectino m0unds: hahaha
i'm 7081mi from hamilton mercutio: bah from home it hit proxy all the time :/ m0unds: or almost 11400km mercutio: this is why i use port 24 :)
m0unds: that's pretty close m0unds: that's if there was a cable draped from NM straight to nz
haha mercutio: so yeah, even if i had gigabit compared to 100 megabit, it'd only be about 20% faster to a location 5 msec away
oh it's 2.5 msec
actually i think gizmo is right about something being broken ***: pseudorandom has joined #arpnetworks
pseudorandom has quit IRC (Quit: Leaving) anisfarhana: Might be off topic, but does gmail have problem yesterday or a day before? mnathani: brycec: what season/episode are you on?
brycec: of breaking bad mercutio: anis: gmail has lots ofp roblems affecting some users but not others and some data centres but not others
so it can easily be fine for other people and broken for you anisfarhana: http://www.wired.com/2014/06/gmail-bug-could-have-exposed-every-users-address/ BryceBot: Wired: "Gmail Bug Could Have Exposed Every User’s Address | Threat Level | WIRED" anisfarhana: Many people complaining about this since yesterday.
But today seem ok.
Yesterday and a day before.
they dont received any emails.
No errors wallshot1: my gmail spam folder has made me aware for years of just how many countless assholes have my email address mercutio: weird anisfarhana: And here too
http://www.dailymail.co.uk/sciencetech/article-2655616/Massive-flaw-revealed-revealed-Gmail-address-EVERY-user-Googles-mail-service.html?ITO=1490&ns_mchannel=rss&ns_campaign=1490 mercutio: my gmail spam mail box only gets false positives
it happens quite frequently anisfarhana: No wonder many friends yesterday dont receive the emails!! mercutio: oh wow
mail just came through for me quickly
i just emadiled myself :0 brycec: Haven't noticed any gmail issues...
mnathani: Don't remember exactly (but Netflix ought to), somewhere around season 3 or 4
(I think I last watched it back in December or thereabouts. But then life got busy, and...) ***: wallshot1 has quit IRC (Ping timeout: 252 seconds)