OSS Line

Telcordia Dips Toe in Cloudy Waters of SaaS for OSS

Is this a precursor to wider deployment of OSS products in the Cloud?

“Telcordia, a global leader in [blah blah etc.], announced today the availability of the Telcordia eLab, a new service that provides integrator partners with cloud-based access to the company's Next Generation OSS portfolio…”

Finally , a serious, all be it small, step towards new deployment architectures and business models for Big OSS. Telcordia’s ‘Next Generation’ software (Granite and Network Engineer) is now easily turn-on-and-off-able for those short term projects that you don’t want to be buying expensive hardware for.

Today, Telcordia eLab is only used for training, demo and testing purposes. But, presumably, there is no reason that, if this is successful, Telcordia couldn’t extend this service to commercial use of the OSS products. Training and testing is the natural first choice though.

And very clever of Telcordia to go for those low-hanging fruit first: It allows them to deliver their training services far quicker, with control over the installation, while reducing the overall total-cost of the activity.

How big is the jump to a full useable installation in the Cloud? Pretty big actually. With this first step Telcordia have technically proven their applications work in a virtual-machine based Cloud environment. But that does not necessarily include the large stack of admin, security and billing infrastructure that is required for a true SaaS offering in the cloud. More significant though, is the step their customers would have to take: Accepting that there are cost-of-ownership benefits to non-perpetual licenses and off-site deployments of business critical software. A leap of faith few telcos have made in the past.

Why Open Source OSS and BSS Products?

With the release of RiverMuse open source 'core'  product a number of people have been asking me about the pros and cons of open source. It's new to our industry, so the implication and benefits (to customers and vendors) are not well understood.

above. thanks to http://www.flickr.com/photos/daviderickson/ / CC BY-SA 2.0

Herewith, my take on it...

Is open source  really new to the OSS/BSS industry?

Well, ok, no. It depends how you look at it. Many large telcos have had open source policies for many years. British Telecom, for example, had a two-pronged approach to IT platforms (including OSS and BSS platforms) that preferred one commercial stack and one linux-based stack. The benefit being:

1. They could buy in, and support,  open source platforms a bit cheaper than commercial platforms.
2. They could use Point 1 to drive down the prices charged by their commercial product vendors.

Use of open source platforms, as in the operating system and some key components like Apache web server, is pretty well established in the wider IT world. It's a low risk strategy.

Open source telco applications are much rarer. Business critical, 24/7, applications even more so.

Many of the big-vendor OSS products were ported to run on Linux a few years ago. Few get open-source-ified above the operating system level. Other open source components, like MySQL databases and jBoss application servers, have no support outside of small, tactical, solutions (until now).

Bits of open source underpinning OSS/BSS solutions are not uncommon; A complete stack of open source, supporting a major OSS process, is something new.

Will the quality be up to business-critical standards?

I expect so. If it's not, that is nothing to do with it being open source.

The fear of open source tends to be:

1. It's written by kids in their bedroom, with insufficient control over design and implementation.
2. The kids aren't even using their own code. They're ripping off IBM/Microsoft/SCO/Whoever. No one's checking the origins. You're going to end up on the wrong end of a lawsuit for using unlicensed code.

This argument has been doing the circuit for many years, mainly being aimed at big targets like Linux and Apache. Both have stood up to the test remarkably well. Quality is at least comparable with commercial products. The big IPR lawsuits have pretty much failed.

Any responsible CIO or IT team should consider these issues, but that should be equally true for software from commercial organizations as well.

A key consideration here is the nature of the OSS/BSS industry, and RiverMuse is a fine example: Bedroom coders aren't writing OSS applications. Professionals, managed within proper businesses, are writing the code.

Outside of the high-profile ones, most long term open source projects are almost entirely coded by a team within, or sponsored  by, one commercial organization.

How will RiverMuse make any money?

There's a few different open source business models, two of which are:

1. Release a fully-featured open source product and generate revenue from services and support only.
2. Release an open-source product with core functionality, sell services and support, but also sell closed-source complimentary products.

Model 1 is most peoples idea of open-source, but it's Model 2 is where the larger open-source companies expect to see revenue. For open-source platforms, like Red Hat Linux, and both MySQL and Ingres databases, the core product is free while the 'enterprise' or 'server' versions are chargeable.

I would expect RiverMuse to be releasing only the core product as free, open-source. In fact, in some recent coverage the term 'core' has been used to refer to the community release.

Announcing an open-source product is also a good marketing move. It generates buzz; I've seen far more blog coverage of RiverMuse than any other recent product launch.

In some cases it may also mean RiverMuse gets a foot in the door. Some OSS deployments exist by evolution rather than creationism. If a few companies start to deploy RiverMuse as a cheap, possibly short-term solution, then they have a stronger starting point for commercial negotiations.

That said, I don't expect many big, tier-1, customers to be won this way. Selling to large organizations is no easier with open-source than it is with commercial, closed-source products.

Should established vendors go open-source?

Sure, why not? Or, if not open-source, then perhaps free core products with at least useful APIs. Some sort of 'community' release.

Apart from it being a useful sales and marketing strategy (see previous answer) it offers other benefits to the vendor:

Allows them to focus on building value propositions for new solutions (catalog, BPEL, service management, data federation, and so on) rather than having to justify the cost of a big bag of pricey products. For example, free inventory but you pay for automation and activation.

It can build a community of companies enhancing the core product or offering value-add functionality. Encouraging more development, such as the Sincera SASI product, would be A Good Thing, extending the OSS vendor’s functionality at little cost to them.

Actually, I'm not fussed as to whether a free or open-source version of the vendor's products are released. I believe that open-source offers limited benefit over free (but closed-source) software in our industry, simply because there isn't actually a community of independent developers wanting to tweak existing products. The need is more about allowing third-parties to configure, integrate and compliment core OSS capabilities. It’s more about allowing add-ons, adapters, interfaces, and cool GUIs to be developed, most of which would be through normal customization methods rather than hacking core code.

Creaner on Cloud Computing

Martin Creaner’s blog teaches us the fundamentals of cloud computing, but where's the TM Forum leadership?

The hype and hope of cloud computing continues. There's an opportunity here, and I'm happy to see more contributions to the debate on how service providers can embrace new architectures. Frequently, though, they lack any meat, or real understanding of what cloud computing actually is.

above. Cloud computing architecture. Click to enlarge

Martin Creaner, President of Tele Management Forum, has recently blogged his two pennies worth. He's somebody senior management listen too, so it's good to see he's written a piece that brings clarity to the whole cloud thing.

Take a look at Martin's article on the TMF blog.

But What About The Cons?

Martin is a little too convinced that cloud computing will be the Next Big Thing. There are plenty of potential 'fatal flaws' in the concept that deter  users today and could easily scupper the movement in the next few years. Just two of them:

No portability. You can't pick-up a Google AppEngine application and reinstall it in Amazon EC2 or Microsoft Azure. Each cloud platform is proprietary. Really proprietary. If you thought supporting applications on Solaris and Windows Server was a hassle, this is nothing compared with cloud computing.

Limited Functionality. OSS and BSS applications are non-trivial. They rely on complex business logic, workflow engines and sophisticated features of relational databases. Most of this is not available in the cloud. You get Java POJOs, a component model, and a flat data store that's got less functionality than Excel.

It may just be a matter of time before these problems are solved. We're starting to see Java support in clouds, and some platforms, like Azure, are expected to introduce proper SQL databases. They better get a move on though...

Here's What the Cloud Can Do For You

The high-light of Martin's piece is his clear explanation of the benefit of cloud computing: You get the resources you need quicker, and pay less up front for them, compared with procuring your own servers and software.

These simple financial savings and time savings are the basis of the cloud proposition. Maybe that's a little disappointing? Were you expecting it to be whizzy-bang technology that revolutionized the way you did X, Y or Z? No. Cloud computing's strength is that the fundamentals are simple, measurable, benefits.

above. Benefits of cloud computing. Click to enlarge

More, Please, Martin

So, Martin's a cloud computing fan. This needs to lead in to some technology leadership from TM Forum. Sorry, but despite Martin's suggestions NGOSS, or any other SOA initiative, is not an enabler of cloud computing. SOA is architecture agnostic, that's the point of it, so having SOA contracts does not make implementing a specific back-end architecture any easier.

The potential, revolutionary, use of clouds is, as Martin surmises, in building a scalable architecture for Telco 2.0-style upstream partners to 'plug-in' to the network and OSS/BSS infrastructure. Partners and service providers can then go mad and deliver the massive suck-it-and-see approach to products that Keith Willetts (TM Forum) has been talking about lately. With the economics and scalability of the cloud, entry costs for new products and partners is small. Scaling-up a surprise success is affordable and, most importantly, possible.

To make this a reality we need APIs, contracts, SLAs and processes that allow partners to run their businesses through service provider infrastructure, much like retail vendors do today on the Amazon platform.

TM Forum would be doing the industry a big favor, while restoring its reputation for innovation, if it delivered a specification for a model Telco 2.0 infrastructure.

Oracle Buys Sun

Yesterday Oracle stepped in to purchase Sun for $7.4bn after IBM failed to secure a deal just over two weeks ago. The fortnight wait has earned Sun a tasty half-billion Dollars over IBMs last best offer.

Oracle’s stake in Java has shot up in the last 15 months. They always had a fairly well received Java J2EE application server, one that had great ‘marketing’ sales, but perhaps not-so-great community acceptance or reputation. Aaaah, it was ok, to be fair. BEA was better though,  so in January 2008 Oracle bought arguably the market-leading Java enterprise platform. And now with the proposed acquisition of Sun, Oracle basically owns Java.

This leaves IBM as the only other major enterprise Java vendor, and Java as a whole looking somewhat less ‘independent’ and less ‘community owned’. In the medium-term the effect will be positive:

• Two choices of vendor both with good, mature, Java platforms.
• Growth of open source Java platforms as a few more people look for an alternative to the market giants.
• Better out-of-the-box functionality, tools, and integration, starting to rival the benefits offered by Microsoft’s single-vendor .NET platform.

However, in the long-term, either IBM or a community initiative may  develop a competitive platform to offer an alternative to the Oracle-centric Java offering. And it could be very well received. I see plenty of scripting/web languages out there that are popular because they do enough and do it quickly (from a development point of view). Java does too much, and as a consequence, development can be slow. An alternative could leap-frog Java’s attempts to resolve the complexity and bloatware issues of J2EE. With a strong commercial backing from the likes of IBM, it would be compelling for both the project-coders and off-the-shelf OSS product-developers.

Yoof Technology - Why The Old Still Own OSS

All those cool technologies out there, I wonder how any young developer will have space for them on their two-page curriculum vitae. And it's not just the quantity, it's the names. Languages, operating systems and middleware used to have nice short names and acronyms. Concise, optimised, no more than four characters long. But now we have “Ruby on Rails”.

above. Choose the right tech for the job.

There's a trend here: The longer the name of the technology the more concise the language and the shorter the development time. At least, that seems to be what's proposed by many new technologies.

The emerging languages  don’t cut the mustard as far as product development goes. I wouldn’t want to support and evolve a product for the next 20 years and use anything but a well established enterprise-grade platform. But that said, sometimes a cool new technology can satisfy immediate needs for a tactical or short-term solution.

The Register ran a report on Twitter's migration to a new system architecture. Reportedly because Ruby on Rails just couldn't hack it. It scaled up to a point then just fell over, famously during Jobs' keynote at a Mac show. To be fair, a massive burst of traffic can bring almost any system down with the right set of conditions. However, Twitter recognised that there was a general scalability issue here, and Ruby was not going to allow their platform to scale along with their business ambitions. So, on the server side, they've swapped Ruby for a little known, concise, language with no proven enterprise track record, called Scala. Okaaaayyy... Let's hope they benchmarked it this time...

Ruby, Scala, Java, ASP.NET, C++ and AMOS Basic all have their place and valid uses. Pick one that’s appropriate for the job, and make sure you prove it can meet your requirements.

above. AMOS Basic – The right architectural choice for bedroom coders in the 1990s.

If you're working on a new OSS system you should quickly narrow the choice based on the fact that there are three categories of technology out there:

Suitable for enterprise-class OSS products. A commercial off-the-shelf (COTS) product's primary non-functional requirements will be portability, scalability, configurability and manageability. Manageability, if it is a real word, means basing the product on technologies that have good support services, release cycles, flexible licensing terms and limited inter-dependencies. You need to be sure your platform components are thoroughly tested on a variety of enterprise servers, and will be supported and maintained for years to come. Broad industry adoption provides some reassurance that the technology will continue to be invested in for the foreseeable future.

Suitable for enterprise-class OSS projects. If you're developing a point-solution, something that has horrendously short delivery timescales and will probably be hacked around every other month, you might as well use any relatively proven technology, whether that be Ruby on Rails or some drag-and-drop Java tool. Rapid development is the priority, provided a stable technology is used, but as its built to a fixed specification and a single known environment the concerns around delivering a managed COTS products are far less relevant. Not every project is led by time-scale. If you're producing something to be used throughout a large service provider that must support network/service evolution then you're effectively writing in-house products.

Other stuff. Some technology is not sufficiently proven to be recommended for anything but the odd proof-of-concept or small tactical intranet application. Other technologies are starting to fall into the gap between enterprise platforms and emerging technology. PHP, for example: Would you start a new project using PHP rather than going with either Java or cool/hip/funky Ruby on Rails? No, PHP is soooo four-years-ago.

Twitter was developed like a small project to deliver a cool little application. I don't think for one minute any product managers sat down to discuss functionality and non-functional requirements. It's just not the romanticised web 2.0 way, is it?

I like new stuff, I really do. But when it comes to using something for an OSS product it better be scalable and robust first. Concise, agile, and rapid would all be on my wish-list, but not near the top. Maybe that's why my CV fits on two pages.

Top 5 OSS Platform Architecture ‘Things’

A little while ago, following someone's good advice, I produced a one-page flyer/data-sheet/brochure outlining how I can help OSS vendors and customers with their platform architecture designs. Check it out.

Offering my services as an independent consultant it's good to have a niche; something only I and a few other people can offer. A rare combination of more common skills is a good start, and in this case my proposition is my experience of OSS solutions and the implications for platform design, hardware, software, security and support. Get your OSS platform wrong and it can cost a fortune and deliver terrible scalability. Get it right, and you can save money on hardware, licenses and actually support your 250 thousand service transactions-per-hour.

That proposition is based on the presupposition that designing and managing your OSS platform is not just some generic IT and DBA activity. Do you think your network inventory database is a self-managing black-box? Do you think your OSS platform requirements are no different from billing or CRM? Is your hardware vendor trying to impress you with TPC-C benchmark results?

OSS systems are unique, and the reality is that few platform vendors, or even OSS vendors, have a great deal of experience of OSS specific platform architecture issues.

Is OSS really special? Need some convincing? Here’s the Top Five OSS-specific platform ‘things’.

Relevance of Synthetic Benchmarks. Assuming your OSS systems are Java based using a relational database, then you may have some interest in the hardware vendor's TPC-C  and SPECj  benchmark results. Both these benchmarks provide some indication of how an OLTP type of application will perform on the specified hardware. They are, however, generic and synthetic; they are not based on OSS transactions. Compared with a typical OLTP application (an online shop, for example) OSS is distinct in having:

• A relatively small number of users (10s & 100s rather than 1000s & 10,000s)
• Relatively sophisticated user interaction with the system
• A high degree of complex automated processes being initiated on the server
• Complex objects defined by highly relational data
• Oh, I could go on and on....

The upshot of this is that OSS architects cannot use benchmarks results as an input in to analysing system performance requirements. Benchmarks are a reasonable indicator of performance relative to another vendor who ran the same benchmark. Such a comparison may be useful, but the best approach is to have a discussion with the hardware vendor to evaluate your OSS specific needs. Which leads nicely on to....

What is a 'Transaction'? It is necessary to have conversation with hardware vendors, and often software vendors, to work out how big a system is needed, and how much money will be spent on CPU licenses. When quantifying the load the OSS system is going to put on the platform it is essential that both parties are talking the same language. Things like ‘number of database transactions' and 'web pages served' might come up. But these are not universal units that measure all computer systems. An OSS database transaction is quite different from that of billing or an on-line shop: OSS data models are more complex and a single business process will read and write far more data rows than those of other types of application.  Similarly, OSS systems may return quite different web pages or rich web UIs. OSS deals in millions of object instances of many different types, while applying complex rules of interdependence between these objects. HR systems deal  in just thousands of objects. Billing and CRM, do have large data-sets but relatively simple data structures. On-line shops, even at the scale of Amazon, have a few thousand product lines and millions of customers but, again, the data objects and relationships are much simpler than the model for something like a customer's SLA-managed, asynchronous broadband service on an overbooked MPLS circuit.

Sizing for Busy-Hour Transactions. At the very start of the OSS design process it is uncommon for architects to be supplied with the data they need to estimate hardware requirements. I've seen many RFPs where the fundamental performance requirement is something like 'support 2 million service creates and re-grades per week'. At best this is a useless KPI, at worst it is misleading because some sales engineer is going to commit his company to supporting those 2 million transactions spread nice and evenly over 8 business hours per day, 5 days per week. In reality it is going to be lumpy. How lumpy is dictated by the OSS applications and the telcos business practices. You need a hardware platform that will support your busiest hour of the week (or month/quarter/year if it's really lumpy), and you need to look at the telco's systems and business practices to provide a realistic estimate.

Read-Only Versus Read-Write Usage Patterns. Most people assume a user with read-only access to a system is a 'light' user producing less load on the system then a fully-fledged read-write user. That's a false assumption for many OSS systems. In OSS, 'write' user activities, like manually designing a circuit or updating service records are relatively slow, updating only a small number of objects. 'Read' activities on the other hand can be intensive, such as running routing reports across multiple network technologies or investigating root-causes of faults. Users running reports can result in database queries across the entire data set, spanning multiple objects. Do not under estimate the needs of your OSS read-only users.

Shared Resources in the Network Model. In OSS, at the service or network inventory level, there will be resources that are either scarce or shared. DBAs see this all the time at the database and hardware level: Database locks, shared memory space, 'hot-spots' of data on a particular hard disc, and so on. The source of these problems can be traced back to the application level. Take, for example, service provisioning. On day one your OSS system might be running perfectly, provisioning PSTN voice lines. On day two it grinds to a halt when you extend the model and processes to provision DSL. Why? Because suddenly interface ports in your network become a shared resource. PSTN terminating copper on an exchange has few, if any shared ports being assigned to the service. DSL will have thousands of users being provisioned on the same data network interface ports on devices, gateways and content servers. The result would be provisioning processes becoming serialized, waiting on locked resources, and it wouldn't matter how many CPUs or disc arrays the platform has because the enterprise-class system is now no more scalable than a desktop application.

There's no theoretical 'perfect' database design. OSS scalability can only be assured by being aware of how a network is structured and how services consume the network, then designing the database and platform accordingly.