I got an official description of what happened July 14 to knock out Tele2's core switch (?) and take down (for some, briefly, in Latvia, well into the night) services to around two million customers in Latvia, Lithuania and Estonia. Despite it being a Swedish-based international telecoms company, the network crash in the Baltics (as far as I could see) got exactly fuck all coverage in the Swedish media, which, like much of Scandinavia, is zoned out in a hammock somewhere or on the beach in Greece before, to use the Latvian expression, the place goes to the Devil's mother.
Basically what happens is this: the core switch, possibly a Nokia MCSI runs on 48 volt DC current, which is fed to the device through a AC/DC transformer getting it from the 220V grid. The transformer apparently can get hot, so it has a "climate control" (read air conditioner or chiller on it). Because the AC/DC transformer is mission critical, the climate control comes with a temperature sensor and some kind of alarm that alerts Tele2 technical staff that the system has failed, but giving them enough time to prevent damage to the transformer. The alarm and the sensor are what may have been the "piecashit" gizmos that ultimately crashed the network. The alarm failed to go off until it was too late. By then, the transformer had overheated and shorted out, knocking out the switch. With no transformer, there was no way to power up the switch until extensive repairs had been made. In addition, very complex systems like mobile phone network core switches do not usually reboot very easily, especially after a power-failure induced crash.
Utility power was still on --Latvenergo's press secretary freaked out a little when the media blamed electricity for the failure and said, rightly, that the electricity from the utility was never interrupted, it all happened inside the walls of the Tele2 facility. So it does look like one fucked gizmo brought down everything..
Except -- was there really no UPS (providing DC electricity) attached directly to the switch to keep it going for a while until the techies fix whatever broke or switch to generators? Perhaps the emergency protocol was to go directly to the generator, forgetting the possibility that the transformer, thanks to some cheapo gizmo, could blow? This is how we learn...
Sporadic commentary on the telecoms and IT market in Latvia and the Baltic States.
Showing posts with label Estonia. Show all posts
Showing posts with label Estonia. Show all posts
Friday, July 15, 2011
Thursday, July 14, 2011
Charlie Foxtrot visits Tele2 in Latvia and the Baltics
As I understand it, you weren't supposed to use obscenity on US Army radios, so instead of saying that something was a clusterfuck, you said Charlie Foxtrot instead. Well, today, and to some extent, still, tonight (2100 local time, July 14), Charlie Foxtrot visited Swedish-owned Tele2 in Latvia and took a chunk out of Lithuania and Estonia as well, knocking a total of well over two million customers off the network (just over a million in Latvia, a million pre-paid users in Lithuania, and the undisclosed prepaid part of a total of 467 000 users in Estonia).
The problems started just after 1400 local time when, according to Tele2's official version, a disturbance in electricity supply took down a major switch. To me, this was an immediate, red-flag WTF?? because mission critical switches have, by default, big motherfuckers of UPS (uninterrupted power supplies) that will keep things going until utility power is restored or switched to emergency generators.
My theory is more that there was some kind of perfect storm event or someone stumbled across a power cable (between the UPS and the Mother of All Switches, if that is possible) causing enough of a power fluctuation to crash the switch at a software level and perhaps fuck up some vital hard disks. That is just my guess.
Business and post-paid customers in Lithuania were unaffected, and in Estonia, pre-paid customers were only down for around 20 minutes, or so the spokesperson said.
Whatever happened, it possibly showed the downside of Tele2 and possibly other operators rationalizing their networks by concentrating services in one switching center (for smaller countries like the Baltics). It appears that prepaid service (billing, switching) were run for all three Baltic countries on servers/switches in Riga. Given how mission critical (more mission critical than if the supporting device and software systems were distributed) the Riga switch is, one wonders how there could be any event involving electric power that could take it down. Some part of the truth may come out tommorrow (July 15).
The problems started just after 1400 local time when, according to Tele2's official version, a disturbance in electricity supply took down a major switch. To me, this was an immediate, red-flag WTF?? because mission critical switches have, by default, big motherfuckers of UPS (uninterrupted power supplies) that will keep things going until utility power is restored or switched to emergency generators.
My theory is more that there was some kind of perfect storm event or someone stumbled across a power cable (between the UPS and the Mother of All Switches, if that is possible) causing enough of a power fluctuation to crash the switch at a software level and perhaps fuck up some vital hard disks. That is just my guess.
Business and post-paid customers in Lithuania were unaffected, and in Estonia, pre-paid customers were only down for around 20 minutes, or so the spokesperson said.
Whatever happened, it possibly showed the downside of Tele2 and possibly other operators rationalizing their networks by concentrating services in one switching center (for smaller countries like the Baltics). It appears that prepaid service (billing, switching) were run for all three Baltic countries on servers/switches in Riga. Given how mission critical (more mission critical than if the supporting device and software systems were distributed) the Riga switch is, one wonders how there could be any event involving electric power that could take it down. Some part of the truth may come out tommorrow (July 15).
Saturday, August 29, 2009
Estonian president Toomas Ilves on digital media
Estonian president Toomas Hendrik Ilves talked about digital media at the Dream on Demand conference at the Estonian TV tower in Pirita, a Tallinn suburb on August 27. The video is a fragment of his speech, in which he also wondered whether the huge tower could become a "gravestone" for conventional broadcast TV (I did not record this :( ). Latvian viewers, please excuse the typographical error in the Latvian title, I am correcting it for inclusion in my nozare.lv blog next week
Monday, August 24, 2009
TeliaSonera offers complete buyouts of Estonian and Lithuanian subsidiaries
TeliaSonera has offered to buy the remaining shares in its Estonian and Lithuanian subsidiaries for just under SEK 5 billion cash. The Swedish telecoms group is offering SEK 3.3 billion for the remaining shares in Eesti Telecom, a holding company that owns 100 % of mobile operator Estonian Mobile Telephone (EMT) and fixed network operator Elion. It is offering just under SEK 1.6 billion for the remaining shares in TEO LT, the Lithuanian fixed network, internet TV, call center and data center services company. Telia Sonera currently holds around 60 % of both companies,
The offer has been criticized as somewhat low and taking advantage of the economic crisis by a Danske Bank analyst. However, this appears to be bottom feeding on the part of the Swedes, not real vulture capitalism (which presumes that the buy-out subject is near financial death).
All of which is fine and dandy, because Eesti Telecom shareholders can get more than 24 % above the last market price for their shares, while TEO LT shareholders are offered a premium of nearly 31 %. The offer also bounced up both the Tallinn (13.3 % at one point) and Vilnius (9.7 % in mid-session) bourses, and even got a 2.7 % rise out of Riga. Which is not to say that it takes much in any of these places.
However, in terms of the Latvian telecoms market, TeliaSonera has ignored its two other (as some European languages put it) daughters or semi-daughters -- Lattelecom (where it holds 49 %) and Latvian Mobile Telephone (LMT) (where it holds around 60%). The reason is that all the rest of the shares are held by the Latvian government --which, to use a term of management disfunction assessment often spoken but seldom written-- cannot unfuck itself to make any kind of decision.
So, while the Estonian government will get a considerable sum for its 24 % in Eesti Telecom, and many private Lithuanian shareholders will at least smile, if not laugh all the way to the bank (the state owns less than 1 % of TEO), the Latvian government will continue to hold its shares (which it could have unloaded for USD 1 billion a couple of years back) like a turkey guarding a shitpile (tītars uz sūdu kaudzes -- is that a Latvian expression?). Or am I wrong??
The offer has been criticized as somewhat low and taking advantage of the economic crisis by a Danske Bank analyst. However, this appears to be bottom feeding on the part of the Swedes, not real vulture capitalism (which presumes that the buy-out subject is near financial death).
All of which is fine and dandy, because Eesti Telecom shareholders can get more than 24 % above the last market price for their shares, while TEO LT shareholders are offered a premium of nearly 31 %. The offer also bounced up both the Tallinn (13.3 % at one point) and Vilnius (9.7 % in mid-session) bourses, and even got a 2.7 % rise out of Riga. Which is not to say that it takes much in any of these places.
However, in terms of the Latvian telecoms market, TeliaSonera has ignored its two other (as some European languages put it) daughters or semi-daughters -- Lattelecom (where it holds 49 %) and Latvian Mobile Telephone (LMT) (where it holds around 60%). The reason is that all the rest of the shares are held by the Latvian government --which, to use a term of management disfunction assessment often spoken but seldom written-- cannot unfuck itself to make any kind of decision.
So, while the Estonian government will get a considerable sum for its 24 % in Eesti Telecom, and many private Lithuanian shareholders will at least smile, if not laugh all the way to the bank (the state owns less than 1 % of TEO), the Latvian government will continue to hold its shares (which it could have unloaded for USD 1 billion a couple of years back) like a turkey guarding a shitpile (tītars uz sūdu kaudzes -- is that a Latvian expression?). Or am I wrong??
Monday, August 18, 2008
No "iPhone 3g Friday"in Latvia
Despite an earlier announcement by Apple officials that the second wave of iPhone 3G launches would start in around 20 countries on August 22, only Estonia's Estonian Mobile Telephone (EMT), a subsidiary of TeliaSonera, has announced that iPhone sales would start on that date.
It looks like Latvia's LMT and Lithuania's Omnitel will launch the newest iPhone at a later date. Apple (together with TeliaSonera) is calling the shots on this and may see tiny, mobile & IT savvy Estonia as the best place to get its feet wet in the Baltics.
The blogs and portals have been full of news about iPhone glitches, and this may be another reason for caution.
In Latvia, LMT might be considering launching an iPhone business package for small businesses using its own Tiešpasts (direct mail) push email platform and wants everything to work right. Perhaps it will equip some key LMT staffers with iPhones as a proof of concept -- if it works for 10 -20 of their guys and gals, it will work for my little company. But that is mere guesswork. The only date one hears for the iPhone 3G in Latvia is the rather subjective "soon", which means different timescales for different folks and situations.
It looks like Latvia's LMT and Lithuania's Omnitel will launch the newest iPhone at a later date. Apple (together with TeliaSonera) is calling the shots on this and may see tiny, mobile & IT savvy Estonia as the best place to get its feet wet in the Baltics.
The blogs and portals have been full of news about iPhone glitches, and this may be another reason for caution.
In Latvia, LMT might be considering launching an iPhone business package for small businesses using its own Tiešpasts (direct mail) push email platform and wants everything to work right. Perhaps it will equip some key LMT staffers with iPhones as a proof of concept -- if it works for 10 -20 of their guys and gals, it will work for my little company. But that is mere guesswork. The only date one hears for the iPhone 3G in Latvia is the rather subjective "soon", which means different timescales for different folks and situations.
Subscribe to:
Posts (Atom)