My longstanding quest for the best tool set for rapid, flexible and powerful development (where for me “powerful” includes robust sql database support, strong dictionary / no
sql data support, access to powerful statistics libraries, machine learning libraries, NLP and other libraries, and support for web apps and restful, back end services), I have settled on Python3, MySQL / Aurora, PyCharm + Sublime, NLTK, numpy, flask and the rest. I am already happy with my productivity, and have recently recognized a need to move from OS-X to Linux for more of the heavy lifting. Which, naturally, means everything is going to AWS…
AWS has improved in a hundred ways in the last three years, and when I was last certified I thought it was the best thing ever. So January 31 I hope to be re-certified but have already begun to migrate my personal projects and infrastructure to the cloud… I expect this to take months as I interleave it with ongoing development and research projects.
All of which is good, and AWS goes a long way for me toward making infrastructure into software. But there is another area I want to understand better and apply in my quest for more efficiency, and this raises the question in the title: Chef or Pup
pet? I wont throw in Ansible or Salt as this article does, and based on some of what I am reading, perhaps my Python penchant might argue one
way, whereas my striving to use workplace-relevant tools and approaches across the board my weight my choice against what is optimums for my current daily workload.
Another factor might be how well AWS integrates with either product, which would weigh more heavily than my personal Python needs, as Ruby and other toolsets are likely to be more important to many of my future clients / employers.
I also should give a big SHOUT OUT (is that big? ) to James and his team at LinuxAcademy who continue, five or seven years in, to innovate and do a fantastic job of providing top-flight hands-on training for AWS / Linux / Azure devs, sysops and architects. Fantastic performance for a small firm that obviously has their priorities right!
But back on Chef vs. Puppet, I will find or create a comparison and figure out if either are going to save me cycles, make me more efficient, or just slow me (and my small team) down!
As I have posted three or four times on the question of how best to create modern, scalable, flexible and robust web applications (here, here and here), I finally decided that Ruby was going to require too much ramp-up time and — when I read about Flask — decided that my tool set for the next quantum of time (five years?) – to replace my joy working with Objective C / iOS / Xcode/ sqlite – would be Python3 / MySQL (Aurora) / Flask (for web framework) / PyCharm, NLTK, Pandas and one day scikit-learn and the rest of it (for now I will stick with Rapidminer and LightSIDE as my black-box for text-based machine learning).
And one last thing: Docker. There are many, many reasons that this presents to me the best-of-breed toolbox of modern application frameworks and tools. To more fully describe — at the “example” level and not at the technology / component level — why these tools, frameworks and yes, deployment choices are perfect for me, there could be no more perfect example than this: I wanted to put together a simple web-based registration scheme: a public facing set of web pages — maybe just one or two — that do a “signup”.
In an relatively few days — or perhaps a couple dozen hours — I was able to create from scratch an entire (simple) web app using Flask, Python3, MySQL (I went with out-of-the-box MySQL rather than Aurora as I will need to learn more about Aurora and that will be fun but not short) — that is now live here (and with a link on my home page).
Just as creating a quick-and-simple one page web site using Bootstrap proved to me the value of that framework — this “demonstration app” has validated my ideas and met my initial goals. I did have a couple of false starts with other technologies, but this one looks good. And like BootStrap, you really only get the benefit of it when working with experts. My lame web site reflects my one day rush to get some pages up — and a pre-packaged template that was free (or cheap enough to be equivalent). But if I wanted to do something serious with that web site, I would hire an expert.
Similarly, web security is awfully complex these days, and a side-benefit (or main benefit for some) from using Docker is built-in “isolation” that is a good starting point for enhanced security, and as such is a foundational component of this new technology stack. But like bootstrap and Aurora, I am going to need to spend more time with Docker to understand it, and for this project, a my Russian friend Yury took care of Docker (and everything else) so that I could get the project done quickly. But I will return to all of this in 2017. And with any luck it all be even more “mainstream” then than they are today. Aurora, for one, seems like another major competitive advantage to AWS, who already has too many to count!
I was very impressed with how quick and easy it was to create a web site with “twitter bootstrap.js” — but only today did a fellow grad student teach me another benefit: support for multi-language web sites. [I found tutorials at https://www.youtube.com/user/wiredwiki very useful, and the one below basically had my site up and running in less than its the running time (it is well over an hour and designed for non-technical novices). The others from this teacher were on individual bootstrap 3 features, rather than the one-stop-shop provided below.]
And Wencheng Hu, an experienced translator and web developer, provided me the key solution element: a complete BootStrap 3 solution for language labels and names, etc. — documented here at http://usrz.github.io/bootstrap-languages/ complete with flags icons (in three sizes) for the 43 languages it supports!
How well this works for a “web app” or even a “blog” on an ongoing basis, i am not sure. probably a blog should just be done as separate blogs for a variety of reasons, but swapping out labels and UI elements in an application is often useful (if, for a variety of reasons, painful). My 8-language support for my “language learning” apps on iOS taught me a lot about the challenges of making an app work for 8 different audiences…
I am very interested to see the final details of my implementation for my new web site. I will post the implementation particulars, as one-page bootstrap web sites are a very handy and useful tool for SMEs, and this neat trick certainly works quite well for a small number of languages. Further, popular and inexpensive hosting services support the necessary tools with ease and by default, by and large, so the follow up post should be quite short. And soon!
After this post, I have continued to get more input and find more articles.
Here are two:
- http://www.rspective.com/blog/berlin-startups-tech-stack, which contains these nice diagrams:
And lots of other interesting graphs…
In the article noted above, I linked here for graphs like this one (where venture funded startups are rated on an “okay”, “good” and “great” scale (grade inflation?) using blue / red / yellow) as reflected in Angle.co standings.
Here are just three (of many many) samples:
And lastly, I saw this piece has is called:
Which web technology should I use?
A handy guide for non-technical founders
Interesting at least as far as learning how some folks see the world, including very rough shorthand impressions of the “pros and cons” of using different tool sets. The primary starting point advocated is ‘do something fast and cheap to prove your concept and that you are willing to throw away” — not something I’ve ever been a big fan of (since using a technology I know best is usually fastest, and building “throw away” code is a bad habit I never developed) but might be right for some people / situations (e.g. when using mid-level or junior people who only write throw-away code…).
Perhaps we are nearly at the point where saying “distributed systems” is as redundant as “software program” always has been, but for the moment I want to consider how a specific issue is heightened by the nature of modern, asynchronous systems, and that issue is “fault tolerance” generally as well as “cascading failures” specifically.
More and more such issues arise — and I was please to read a particularly lucid explanation of a popular and important design pattern used in many solutions: the Circuit Breaker pattern. On Martin Fowler’s blog — haha. I was kind of surprised by that — but only because I don’t google interesting problems in architecture and design nearly as often as I’d like.
I can’t add any value to what he’s written here, so instead i will just quote briefly:
The basic idea behind the circuit breaker is very simple. You wrap a protected function call in a circuit breaker object, which monitors for failures. Once the failures reach a certain threshold, the circuit breaker trips, and all further calls to the circuit breaker return with an error, without the protected call being made at all. Usually you’ll also want some kind of monitor alert if the circuit breaker trips.
There are added bits about adding a capability to attempt automatic reset (at some specified interval) and discussions of other real-world refinements (e.g. different thresholds for different sorts of errors), but a hallmark of this sort of writing is that, at least for most of its intended audience, a simple example provided in detail, and pointers to additional kinds of flourishes and add-ons, is really all that is needed.
Check it out! And if you googled this topic, doubtless you have read or seen something about NetFlix’ Hystrix, which says on that getHub landing page:
Hystrix is a latency and fault tolerance library designed to isolate points of access to remote systems, services and 3rd party libraries, stop cascading failure and enable resilience in complex distributed systems where failure is inevitable.
It is a java implementation; there are other articles linked here and links to alternative Circuit-breaker patterns in Ruby, Java, Grails Plugin, C#, AspectJ, and Scala listed at the bottom of the Fowler blog post.
I almost decided to simply add a series of “update” links to my prior post on my quest to understand the current state of “modern web app development”, and in particular, tools, frameworks and environments of choice. Now as ever, many technologists live in their silos, and while the best and most enlighten attempt to glance across at what others are doing, keep up with new ideas and tool kits, the reality is that most top-tier, hard working, delivering-in-a-crunch developer / architects are almost always too busy to do as much of this sort of thing as they’d like. And while I am not one of those, I am also a busy person with many balls in the air, ideas kicking around, prototypes-in-process, apps-in-process, with a real job and various research projects…
I will rectify this with future posts about specific combinations of web app development tools and platforms, where there are available tutorials and low barriers to adoption. And after about five of these, i hope to have come up with a more coherent scheme for comparison.
If you have deluded yourself you have a great idea and a solid plan for its realization, and it involves “the web” — and how could it not? — How to start, and what to use?
At some point you might actually be thinking “which JS Framework is the one to bet on?
Other than my sympathy, i can also offer this neat URL to tell you so much you didn’t already know about who is using what: click here to see the google trends view of Js frameworks… now.
Is this fair? is it right? is it that simple? I stared at this for quite a while, before realizing what was missing (I think — this is not an area I know much about!) Node.JS anyone? Then change the timeframe, change the regions, etc etc. Does it really mean anything? I suspect it does… but then, not really so much that is useful in the particular even if interesting in the general case.
Now back to those other big questions — RoR? Python for everything? Go? It seems Django / Phython or RoR coupled with JS framework might be good if, say, your UI needed to show markup language effects in a WYSIWYG editor for some reason…
So much to investigate… I think I will just ask the experts!
Update: Some interesting data on what high growth startups are using: here (based on Anglelist data, so…).