My longstanding quest for the best tool set for rapid, flexible and powerful development (where for me “powerful” includes robust sql database support, strong dictionary / no
sql data support, access to powerful statistics libraries, machine learning libraries, NLP and other libraries, and support for web apps and restful, back end services), I have settled on Python3, MySQL / Aurora, PyCharm + Sublime, NLTK, numpy, flask and the rest. I am already happy with my productivity, and have recently recognized a need to move from OS-X to Linux for more of the heavy lifting. Which, naturally, means everything is going to AWS…
AWS has improved in a hundred ways in the last three years, and when I was last certified I thought it was the best thing ever. So January 31 I hope to be re-certified but have already begun to migrate my personal projects and infrastructure to the cloud… I expect this to take months as I interleave it with ongoing development and research projects.
All of which is good, and AWS goes a long way for me toward making infrastructure into software. But there is another area I want to understand better and apply in my quest for more efficiency, and this raises the question in the title: Chef or Pup
pet? I wont throw in Ansible or Salt as this article does, and based on some of what I am reading, perhaps my Python penchant might argue one
way, whereas my striving to use workplace-relevant tools and approaches across the board my weight my choice against what is optimums for my current daily workload.
Another factor might be how well AWS integrates with either product, which would weigh more heavily than my personal Python needs, as Ruby and other toolsets are likely to be more important to many of my future clients / employers.
I also should give a big SHOUT OUT (is that big? ) to James and his team at LinuxAcademy who continue, five or seven years in, to innovate and do a fantastic job of providing top-flight hands-on training for AWS / Linux / Azure devs, sysops and architects. Fantastic performance for a small firm that obviously has their priorities right!
But back on Chef vs. Puppet, I will find or create a comparison and figure out if either are going to save me cycles, make me more efficient, or just slow me (and my small team) down!
A new piece from Renee Dudley and Reuters. Some I know were disappointed by “yet another multiple choice test”, or with the continued dominance of “the number-two pencil”, while others were nevertheless satisfied with this somewhat new version of the test if only for a couple (handful?) of content-related changes they were happy to see. Some less so. Some were enraged.
After this post, I have continued to get more input and find more articles.
Here are two:
- http://www.rspective.com/blog/berlin-startups-tech-stack, which contains these nice diagrams:
And lots of other interesting graphs…
In the article noted above, I linked here for graphs like this one (where venture funded startups are rated on an “okay”, “good” and “great” scale (grade inflation?) using blue / red / yellow) as reflected in Angle.co standings.
Here are just three (of many many) samples:
And lastly, I saw this piece has is called:
Which web technology should I use?
A handy guide for non-technical founders
Interesting at least as far as learning how some folks see the world, including very rough shorthand impressions of the “pros and cons” of using different tool sets. The primary starting point advocated is ‘do something fast and cheap to prove your concept and that you are willing to throw away” — not something I’ve ever been a big fan of (since using a technology I know best is usually fastest, and building “throw away” code is a bad habit I never developed) but might be right for some people / situations (e.g. when using mid-level or junior people who only write throw-away code…).
Modern Application Architecture:
Principles for Cloud-based Solution Design
I’ve been working on a small white paper; a monograph of sorts. But then that overstates it. I am I would say writing a short paper to present and explain the primary elements of “good” solution architecture in the (current) modern era. The scope of this perspective on what is “good” is particularly for mission-critical, production applications supporting real programs or activities that are expected to work reliably and continuously with minimal handholding. It is informed by the advent of mature but still advancing cloud-computing building blocks, and by in increasing need in every enterprise for solutions where economic scalability is often the primary “reach” for solution architectures that must also serve the lessor gods of “high performance”, “high availability”, “graceful degradation”, “seamless failover” and “automatic recovery / restart”.
At this point I’d say the primary elements are:
- Memory and Caching
- Parallelization & Partitioning
- Data Replication
- Event-Driven Processing
- Distributed Processing
Of course, all of these are inter-related. In fact “partitioning” is at the heart of everything, and intertwined with “parallelization”. And most of these principles apply at various levels — from the micro to the macro.
And I have thought about organizing the work to show how these principles were employed in 1999-2000, at great cost and effort, and how they can be realized using cloud-based (AWS in this instance) building blocks in 2014, at almost no cost (or no “incremental” cost, and in some cases, due to elastic scalability, lower cost).
I am not sure if this notion of “how we did it then” and “how you do it now“, on a design-principle-by-design-principle basis, is going to really illuminate the problem and the brilliance of the current solution set building blocks as brightly as I’d hoped. But so far I think the problem is my writing.
In fact it needs so much work I am not sure I will ever get to it again. So I will link to it here (pt1 and pt2) and hopefully fix it or take it down before too long… [new version linked in subsequent post]
I started my career at 16 as a FORTRAN programmer at NASA GSFC in the summer of 1974. My first technical paper called “Development of [the] White Sands Missile Range Atmospheric Density Profile”, (here) was delivered “9 August 1974” (note the “Buy US Savings Bonds” admonition on the letterhead) which turned out to be emblematic of my future work: I released code and data; I wrote up exactly how I’d interpreted the need and how I had done the job; I took a conservative (“stacking worst cases”) approach; and (already!) was somewhat pedantic if not also long-winded — and interested only in serving the need and solving my problem, and learning how I might have done it better or to help others improve it.
I think there are several reasons that I enjoy bringing up my longevity and length of practice in an industry, and in an era for an industry, that often seems to celebrate youth and inexperience, and show bias against experience and depth. I won’t waste any time decrying the “agism” that inflects many corners of TechLandia. For the most part I think it is highly concentrated to a particularly venal and comic degree in a particular slice of an oddly un-common sub-class of American business (as nicely captured by TNR here). Otherwise I think it is mostly similar to the broad reality of business culture (at least in the USA). That is, it seems mostly to reflect attempts to attract a particular demographic (younger = cheaper) that some lame “recruiter” folks (if not an infantile CEO) thinks will be attracted to “beer bash Fridays” (groan). But more often it’s a bit more subtle — as when “ServiceNow” so boldly put it — “We want People Who Have Their Best Work Ahead of them, Not Behind Them”. (Hmm. Are there really many ways to read that? Well, they took it down and their stock price is recovering nicely…) (And “move the dial” does seem an improvement, in any event.)
My reasons are mostly that, in professional settings, I am (like the software engineering community I grew up in) an unusually direct person (compared to the general public, and particularly to the “old-boy, live and let live, rank-has-its-privileges” world of many corporate empires); that it shows that, at least in my mind, this experience is to be celebrated and ads gravity to my views and judgments; and to help surface, sooner rather than later, a reaction from others that will give me a read on their own take on this issue (people that are most dismissive of actual experience are surprisingly candid in making it known).
But perhaps another reason is humor: I think “old people” have many odd and funny habits, and are often visibly connected to a world that no longer exists or at least has ceased to exist for half or more of the population. And so by acknowledging my
(near) octogenarian credentials elder status, I am then free to point these things out and laugh at them with, and not at, my less-technology-saavy colleagues of any age. (Well, I am still more than two decades away from being able to claim “octogenarian” status, but for the most part, the real divide in my experience is between people who grew up with computers, and those that did not; or those for whom computers are natural extensions of what they do and how they think — and those for whom is is an afterthought or inconvenience. Or more simply, perhaps just between people who don’t do text messages, and everyone else.)
That said, todays column is just to celebrate forty years in the exciting world of software and of using “computers” to change how things work. A future note will perhaps recount my favorite living-in-a-past-that-doesn’t-exist observations. Hopefully more amusing than people who say “dial” for calling a phone number, and more novel / less annoying than folks who give you an address, then try to also give you detailed, “turn by turn” directions…
Hope the next forty are as interesting!
p.s. If I had waited two more months, I could have led with “It was forty years ago today…” but since the lyric is only “twenty”, I opted (in a rare exception) to forgo the combination bad pun / classic rock reference….
While Amazon Web Services cloud computing platform remains a front-runner of gigantic proportions, ahead of the pack by a margin not seen since Apple’s iPad jumped ahead of the entire consumer electronics industry, forcing the competition to spend a good two years playing catchup. Now, with the entry of Google into the “cloud platform” arena, perhaps an era of real competition… is getting closer!
While I used Google App Engine long before I first logged into an AWS console, I must admit to a) being both a huge fan of the suite of services (and great usability via a range of customer-friendly control panels, command sets and API’s) that AWS offers, and b) a Certified Amazon Web Services cloud Solution Architect — so I can’t hold myself out as being totally objective. That said, while I’ve had day-one orders for multiple versions of the iPad, iPhone and other magic products, I have also owned (and built apps for) any number of Android-flavord platforms and devices, and yet think I am pretty rational about the strengths and weaknesses of the iOS / Xcode / Objective C toolset vs. it’s Android cousins (Eclipse, particularly, with the appropriate pug-ins). [I have not yet tried the new, improved Intelli-J ide for Android which I hear addresses some of my particular complaints. Look forward to that when time permits!] So not being religious about operating systems and languages in my youth, I remain open to platforms, frameworks and tools — based on suitability to purpose — for technology generally.
So I will remain objective in my approach to Google’s new services as I continue working with AWS, and present below some initial comparisons — here – with help from a post by Praveen Kumar Muppala, as referenced below – which I have borrowed from heavily for content while adding and expanding (and editing) the information a bit.
Praveen makes the case that Amazon AWS remains the leading Infrastructure As a Service (Iaas) Cloud Platform provider by identifying 10 key reasons. I’ve presented some summary information below, and then his 10 points with some light editing – the original post can be seen here.
1. Cloud Services & Features
Amazon AWS has more services and features. As of now, Amazon AWS has 27 services while Google Cloud has less than 10 major services. AWS has a more comprehensive offering, which gives it a clear edge in complex, large scale cloud efforts. I’ve show the products / services of the two companies below for comparison.
Google Cloud Platform:
It is interesting to look at some of the details, as Google seems to be evolving toward many of the same use cases as AWS. It is not surprising then, that Google Cloud Platform service offerings that are comparable to AWS offerings in the areas of storage options, load balancing, auto-scaling groups and even DNS begin to mirror and AWS’ options and capabilities even more closely
2. Regions and Zones
Amazon AWS has 8 Public & 1 US Government Cloud Region, and a preview of a tenth region as of today. [The list is: N. Virginia, Oregon, N. California, Ireland, Singapore, Tokyo, Sydney, São Paulo, GovCloud and Beijing(preview)]. And there are 2 or more zones within each of those various regions — up to 5 (see chart below). This allows multi-zone deployment within each region to facilitate high availability and redundancy for deployments of almost any size. Google currently shows two zones in each of two regions. Customers may find that AWS can give them lower latencies for specific target geographies with this greater number of choices.
AWS vs. GCP
3. Supported Operating Systems
Praveen notes that “Amazon AWS supports ten different major Linux and Windows operating systems / versions, while Google Compute Engine supports 4 different Operating Systems”. What I have seen is that AWS supports several major Linux distributions plus their own AWS-flavored machine images for easy deployment and use with AWS services, but also multiple Microsoft Windows server machine images as well. With Google Compute Engine, I see (here) support for two Linux flavors (Debian and CentOS) but none (yet) for Windows. So if you need Windows Server (Microsoft Windows Server 2003 or 2008), it seems the only options are Microsoft Azure and Amazon AWS. AWS offers many Linux options, including “Amazon Linux” (which has some Python AWS management tools and other goodies pre-installed) in addition to preconfigured systems such as Red Hat Enterprise Linux, SUSE Linux Enterprise Server, Ubuntu Server 12.04 and, with some requirements, your own customized Linux Kernel (see here). In addition, the huge library of pre-configured machine images shared by customers would seem to give a huge edge to Amazon for many small and medium sized users that might find the right pre-configured image to be a huge time saver.
4. Hosted Hardware Maintenance
It looks as if AWS and GCP have taken very different approaches to hardware maintenance. I’ve not studied this in all it’s depth, but Amazon seems to perform such maintenance based on when the underlying hardware is available, whereas Google in some instances can seamlessly move your images and jobs elsewhere, do hardware maintenance, and move the workload back without customer impact (other than transitional performance issues) which is good…. but in other cases, Google’s scheme seems to require instances to be terminated or auto start / stopped to make way for a maintenance interval. It’s not clear but there may be no way to avoid some level of forced unavailability on a quarterly basis. This area I need to study more (Praveen’s post seems to declare GCP unsuitable for some apps).
5. Instance Types
Amazon AWS has a large variety of sizes and types of instances, with different combinations of vCPU, RAM, Network, I/O etc. It supports around 30 instance types RAM ranging from : 0.6GB to 244GB and CPU ranging from : 1 vCPU to 32 vCPUs. With Google Cloud Compute, instances available include 12 different instance types RAM raging from : 0.6GB to 52GB and CPU ranging from : 1 vCPU to 8 vCPUs. Compare Google here with Amazon here.
6. Spot Instances
Paraphrasing Praveen, with “Amazon AWS, Spot Instances are spare Amazon EC2 instances for which you can name your own price based on historic data. This can help Customers run large number of stateless EC2 instances” at a reduced price. Google Cloud Platform does not yet have a comparable feature in place.
7. Reserved Instances
Praveen also notes that “Amazon AWS Reserved Instances provides great savings on instances based on instances that run for a period of time with known utilization patterns. Using reserved instances can yield up to 65% savings”, and again, Google Cloud Platform does not yet have a comparable feature in place.
8. Support Charges
Amazon AWS has basic Developer Support starting at $49 / month. Business Support and Enterprise pricing starts from $100 per month, with cost a function of monthly AWS usage (see here). with Google Cloud Platform, basic support (silver) starts at $150 / month and business (Gold) support starts at $400 per month, increasing for Enterprise (Platinum) Support with pricing negotiated based on a variety of factors (see here). For smaller and conscious enterprises and those dipping their toe in the water but wanting some level of official customer support and responsiveness, AWS is a cheaper alternative (on this basis).
9. User Management
Praveen makes a good case for Amazon when he notes that “AWS is supports creating users using LDAP services which allows centralized user management in many organizations already using LDAP. Amazon also supports creation of unique AWS users for your account based on simple username / password pairs, without the need for unique email ID accounts. Google Cloud Platform is expecting a Gmail ID for unique users, giving AWS more flexibility in dealing with managing users accounts.”
10. User Permissions to the Resources
Finally, Praveen notes that “Amazon AWS has 9a sophisticated IAM (Identity and Access Management) capabilities which allows AWS resources to have be controlled via permissions via resource or feature per user. Google’s scheme has but three permissions (Owner, Editor and Viewer), and no ability to assign permissions at the resource level.” AWS therefore enables far greater control and management of access to features and resources.
Thanks again to Praveen Kumar Muppala for highlighting some important distinctions between AWS and Google Cloud Platform. (Praveen’s original post is here.) I won’t be surprised if some of these gaps close more quickly than did the “iPad” vs. Android Tablet gap, but then some of these features are quite complex, robust and technically sophisticated — and Amazon’s huge many year head start may be hard to erase in mere months.
From “All Things D”: interesting glimpse of Tim Cook, giving a speech at an award ceremony at the UN.
Worth a listen. Hope it’s embraced everywhere, including Alabama — to which I have not returned since 1968.
A bit more and commentary from Ina Fried / WSJ see here.