My field is web development, and, to be honest, the most exciting thing going on is that more people are starting to complain about the complexity of development. Hopefully this will lead to people slowing down and learning how to write better web software.
As an example, one survey (https://ashleynolan.co.uk/blog/frontend-tooling-survey-2016-...) put the number of developers who don't use any test tools at almost 50%. In the same survey about 80% of people stated their level of JS knowledge was Intermediate, Advanced or Expert.
Yeah, this is one area where webdev is way behind other fields, and i think we're going to see lots of new tooling in this area soon.
We're currently working on a way to help devs test web app functionality and complete user journeys without having to actually write tests in Selenium or whatever. The idea to is let devs write down what they want to test in English ("load the page", "search for a flight", "fill the form", "pay with a credit card", etc), then we'll use NLP to discern intent, and we have ML-trained models to actually execute the test a browser.
You can give us arbitrary assertions, but we also have built-in tests for the page actually loading, the advertising tags you use, page performance, some security stuff (insecure content, malware links). At the end we hand you back your test results, along with video and perf stats. It’s massively faster than writing Selenium, and our tests won’t break every time an XPATH or ID changes.
>tries to do so by using an imprecise, context-dependent language designed for person-to-person communication to instruct a machine
???
Selenium is its own can of worms, but it absolutely sounds like you're using the wrong tool for the job here. The problem stopping people from writing browser-based tests is not that people can't understand specific syntaxes or DSLs, it's actually the opposite: people don't have a good, reliable tool to implement browser-based testing in a predictable and specific way that does what a user would intuitively expect.
Selenium fails here because it has to manage interactions between browsers, because selectors are hard to get right on the first try and continually break as the page's format changes, because JavaScript can do literally anything to a page and that is really hard to anticipate and address reliably from a third-party testing framework like Selenium, especially if components are changing the DOM frequently, etc., because Selenium is subject to random breakage at the WebDriver layer that hangs up your (often long-running) script, and so on.
Whatever the right answers to a next-gen Selenium are, attempting to guess the user's meaning based on Real English by something that is itself an imperfect developing technology like NLP is pretty obviously not the correct toolkit to provide that. Remember, a huge amount of the frustration on Selenium comes from not having the utilities needed to specify your intention once and for all -- the ambiguities of plain English will not help.
If your thing works, it will have to end up as a keyword based DSL like SQL. SQL is usually not so scary to newcomers because a simple statement is pretty accessible, not having any weird symbols or confusing boilerplate, but SQL has a rigid structure and it's parsed in conventional, non-ambiguous ways. "BrowserTestQL" (BTQL) would need to be similar, like "FILL FORM my_form WITH SAMPLE VALUES FROM visa_card_network;"
The biggest piece that's missing in Selenium is probably a new, consistent element hashing selector format; each element on the page should have a machine-generated selector assigned under the covers and that selector should never change for as long as the human is likely to consider it the "same element". The human should then use those identifiers to specify the elements targeted. I don't know how to do that.
The second biggest piece that's missing from Selenium is a consistent, stable WebDriver platform that almost never errors out mid-script; this may involve in some type of compile-time checking against the page's structure or something (which I know is hard/possibly impossible because of JS and everything else).
Totally agree with this. The concerning part for me is that ML is about making a "best guess" given some data. This means that your tests may pass one time and fail another - inconsistent tests aren't tests at all.
Your post gave me deja vu to an automation workflow I cobbled together a few months ago, which I found wonderfully productive for steering a bot: Vimperator keybindings. Selenium can use most of them right out of the box. It's a terrific navigation layer. For instance, pressing "f" enumerates all the visible links on the page and assigns to each one a keybinding. The keybindings are displayed in tooltips and can be trivially extracted with CSS. You can keep sending keys to the browser, and only links that contain the anchor text remain in the set of candidates. Of course the "hashing" of links to keybindings is completely relative to the viewport, so this won't satisfy you completely. But it was an idea I had randomly one day, as an alternative to the trapeze act of navigating through the boughs of the DOM tree, and lo and behold it worked nicely.
Sounds intriguing. There are a few tools for recording interactions with a webpage in order to replay the actions as a test (Ghost Inspector, Selenium IDE, etc) but they tend to be pretty horrible. I've been working on my own as a Chrome extension for a little while. What you're building sounds really interesting though, especially if it can deal with complex Javascript apps. Anything that can make developers more inclined to test things is a good thing.
Alternatively, one of the consequences of React is that the front-end can largely be unit-tested. You can at least get a pretty good idea that the page will render what you expect if it gets the data you expect.
And whether or not it gets that data is a unit test in another place.
I'm not a huge fan of React, or javascript, but having been forced to work in it, this is one of the wins.
You could test the front-end before react though? Also, one can still have a God component. I don't think React changed anything from a testability point of view. Well written modular code is well written modular code.
Frameworks like Knockout have been around for quite some time now. You don't have to use React to not depend on the DOM. There are many alternatives to "jQuery based front-end development" that's not React. Aurelia, for instance, happens to be an amazing framework in my opinion that's also highly testable and that's not React. Like I said, modular code is modular code. You can write good, modular code with just require js modules, and you can also write terrible monolithic React components.
Testability isn't the domain of the view layer.
Abstracting the DOM into a declarative DOM is great for performance, but doesn't lead to necessarily more testable code.
We are using ghostinspector mainly for it ability to compare screenshots between runs. I think the future of testing will be apps like this that don't require you to specify every little div but just record your actions and play them back and catch differences. Right now ghostinspector only takes a screenshot at the very end but they are adding a feature where you can take a shot anytime. As these apps get better at knowing what matters and what to ignore - all the better.
Yeah, I've looked at all the current tools and there's basically two types (other then just writing straight Selenium):
- Test recorders that aren't a great experience and output incomprehensible, brittle tests.
- Test composers that I can best describe as 90's SQL query builders for Selenium.
Complex JS apps are still a challenge for us (especially with some of the WTF code we come across in the wild), but we have a strategy in the works for them. We're still pre-release though. If you're interested, send me an email (donal@unravel.io) and I'll add you to our alpha list.
From reading the web page, I think an Unravel user won't need to specific, pre-defined language, in which case it's very deliberately not a DSL like Gherkin/Cucumber.
I've been interviewing junior/intermediate frontend candidates for the past few months now. 90% don't use any test tools, and their biggest complaint is their current employer forcing a new framework/library for the sake of being bleeding edge. While interesting to them, it turns out most of them really just want to see what they can do with vanilla JS.
Why you expect that junior developer (someone with very little or no development experience) will use a test tool or any other development technique? I expect that junior developer in software field should be able to program only.
I expect that junior developer in software field should be able to program only.
That is very often the case. It needs to change. Testing is a part of software development, and anyone who writes software should be aware of it. I feel the same way about documentation. And requirements. You can't write good software without knowledge of the processes that surround development. It isn't enough just to be able to write great code.
Maybe. Personally, I've come to think that you need the right tool for the right job.
If you spend more time writing / running tests that you would fixing the bugs they find, you may be doing it wrong. If you're writing documentation no one will read, you may be doing it wrong.
They clearly do have a place though. As for maintaining a set of requirements... I appreciate there must be some environments where what is required is well understood and relatively stable. I'm not quite sure if I should look forward to working in such a place or not!
I appreciate there must be some environments where what is required is well understood and relatively stable.
Actually there isn't. Every project, no matter how it's managed, changes as it goes on. It has to because you learn and discover things along the way. That's why maintaining and understanding project requirements and how they've changed is incredibly important. If you don't keep on top of them then you end up with a project that wanders all over the place and never finishes. Or you build something that misses out important features. Or the project costs far too much. Requirements are not tasks, or epics, or things you're working right not. They're the goals that the tasks and epics work towards.
(My first startup was a requirements management app.)
> If you spend more time writing / running tests that you would fixing the bugs they find, you may be doing it wrong.
Why should those 2 activities be compared? They do not compare: writing/running tests is about discovering the bug, not fixing it. You still need to fix it after you have done your testing activity.
The time spent writing/running tests should better be compared to the time spent in bug discovery without tests, i.e. how much you value the fact that your users are going to undergo bugs, what the consequences of the users hitting bugs are, what the process to report them is, etc.
You're right, unless you're at an extreme (zero automated testing, zero bugs found in the wild) it's much more nuanced as to what the balance is (or should be), but there is a balance.
I think there's somewhat of a gap between "junior" and "junior/intermediate", but given my understanding of university / bootcamp curricula, it's probably both the case that it's unrealistic to expect junior devs to have meaningful testing experience and that it's essential to make sure that potential hires have some awareness / positive attitude toward testing as part of software development.
>Why you expect that junior developer (someone with very little or no development experience)
I don't think thats the definition of a junior developer. Test tools are apart of building software, you should be hiring devs that have created projects that use tests of some sort, if not with the technology you're using.
>I expect that junior developer in software field should be able to program only.
I don't know how you can have little to no dev experience and know how to program.
Development and programming require different skills.
Developer need to know development cycle, automated testing, continuous integration, software life cycle, ticketing systems, source control systems, branching and merging, cooperating, etc.
Programmer need to know programming languages, patterns, algorithms, computer internals, effectiveness, profiling, debugging, etc.
Junior developer (in software filed)) has no or little experience in development, so junior developer is almost equal to a programmer, which causes lot of confusion.
Because we assume that developers are trained professionals, presumably with a CS or software engineering (or both) degrees, and that they've been properly trained in software development - which puts testing front and center.
Computer Science has absolutely nothing to do with software testing. Your software engineering classes will teach students about unit tests, but not much more.
If by 'testing' you really mean 'unit testing', as I suspect most junior engineers who claim testing experience do, then hope is already lost. The one saving grace is that there is enough churn in webdev that nothing lasts long enough to reveal how fragile it is.
Not if they take a good class in Test-driven development (TDD) - which I would recommend to students. The "science" behind it will outlive the practice churn.
Of course, if the will take a good class in test driven DEVELOPMENT, they will be developers. Development (problem solving with goal to create and support a product) is not same as programming (creating instruction for computer to do something).
Really? Where I work we expect knowing how to test code and being careful and incremental in our junior candidates more than anything. It's easier to teach someone how to code better than let them send anything to production with 0 tests.
To me as a web developer, the most exciting new development is react native (not react itself) - it's redefining the border between web and native apps in a way that cordova and xamarin never did.
As a web dev, the most exciting development I see is the rise of progressive web apps and a shift away from native apps in situations where web-like experince is more appropriate.
That said, I'll be thrilled if React Native gives rise to higher quality apps in situations where a native app is unavoidable (e.g. my bank's app).
I'm actually hard-pressed to think of any non-gaming interface that is better suited by a native app than a web app in 2017.
Five years ago native apps made a level of UX possible that was unheard of on the web, to say nothing of mobile. But today not only has HTML/js closed the gap, but whiz-bang native animations aren't impressive just on account of being novel anymore.
I'm feeling that React Native is just another artificial constraint we developers have to deal with. I would have preferred it if Facebook (and/or Apple, Google) would have pushed WebApps more instead. A web browser, with the right amount of love, would be more than capable of doing the stuff that React Native can do.
Google still is! Progressive web apps now have the ability to be installed "natively" on Android devices [0], meaning they show up in the app drawer like any other app rather than being limited to a home screen icon.
I think these are the future. Once they catch on with mainstream consumers, native apps won't stand a chance against the convenience of simply visiting a website to install/use. Plus, on the developer end, we finally have a true "write once, run anywhere" situation that doesn't involve any complex toolchains or hacky wrappers.
There's a market imo for a full solution that includes front end and entire backend including deployment, seemless scaling, seemless upgrading, seemless backups, seemless local dev, seemless staging, etc.
99% of web apps need the same features but most of this is still up to manually rolling your own.
I should be able to clone some repo, enter some DO/AWS/GOOG keys and push.
I've been using and loving Zappa[1] lately. Basically it lets you seamlessly deploy a flask app to AWS Lambda -- that solves your deployment, scaling, upgrading, staging, backups, etc. And local dev is just running the flask app locally.
I think the most exciting trend in web development is the rise in popularity of functional programming styles in front-end frameworks.
This makes the complexity problem much easier to solve, as the code is (should be) less likely to cause an unanticipated mutated state which can't be easily tested for.
I remember this same "rise in popularity" in 2005. Every few years functional advocates get all excited (last time was F# support in Visual Studio, all C# programmers were going to switch, naturally).
I suspect the reality is a small subset of programmers think functional programming is amazing and everyone else hates it. You might think it reduces complexity, but a lot of people feel it reduces comprehensibility.
> I suspect the reality is a small subset of programmers think functional programming is amazing and everyone else hates it.
Yep. It's not that I hate it, I just don't like it. The thing is that the functional-praisers are much more vocal about how they love it whereas people who write imperative do not care much about Haskell.
We are happy with LINQ and that's all 99% of us wants/needs.
Would you mind sharing an example or pointing me to a good guide that explains this concept? How does functional programming make a problem less complex than OO or imperative? I've heard this a couple of times, but the intuition has never quite clicked for me.
Its state. Functional programs tend to have less state as their output is the same for some given input. With things like jquery you quickly introduce state, say is some dropdown open, which your next function will have to check is true or false before proceeding. And so on.
I'll give it a shot; functional programming style--some languages enforce it, some languages merely have features that allow for it if the author is disciplined enough to do so (JavaScript is in the latter camp)--generally eschews mutable state and side effects, i.e. a variable `foo` that is declared outside of the function cannot be altered by the function. Some "pure" functional languages restrict all functions to a single argument. This may feel like an unnecessary constraint (and opinions vary), but one thing that can't be denied is that it keeps your methods small and simple; in any case, it can be worked around by applying a technique called "Currying"[1], which is named after the mathematician & logician Haskell Curry, not the dish (also the namesake of the eponymous functional programming language [the mathematician, not the dish]).
Because nothing outside of the function can be changed, and dependencies are always provided as function arguments, the resulting code is extremely predictable and easy to test, and in some cases your program can be mathematically proven correct (albeit with a lot of extra work). Dependency injection, mocks, etc are trivial to implement since they are passed directly to the function, instead of requiring long and convoluted "helper" classes to change the environment to test a function with many side effects and global dependencies. This can lead to functions with an excessively long list of parameters, but it's still a net win in my opinion (this can also be mitigated by Currying).
A side-effect (hah) of this ruleset is that your code will tend to have many small, simple, and easy to test methods with a single responsibility; contrast this with long and monolithic methods with many responsibilities, lots of unpredictable side effects that change the behavior of the function depending on the state of the program in its entirety, and which span dozens or hundreds of lines. Which would you rather debug and write tests for? Tangentially, this is why I hate Wordpress; the entire codebase is structured around the use of side-effects and global variables that are impossible to predict ahead of runtime.
There is much, much more to functional programming (see Monads[2] and Combinators[3]), but if you don't take away anything else, at least try to enforce the no-side-effects rule. A function without side-effects is deterministic; i.e. it will always give you the same output for any given set of inputs (idempotence comes for free). Because everything is a function, functions are first-class citizens, and there are only a few simple data structures, it becomes easy to chain transformations and combine them by applying some of the arguments ahead of time. Generally you will end up with many generalized functions which can be composed to do anything you require without writing a new function for a specific task, thus keeping your codebase small and efficient. It's possible to write ugly functional code, and it's possible to write beautiful and efficient object-oriented code, but the stricter rules of functional style theoretically make the codebase less likely to devolve into incomprehensible spaghetti.
Manning Publications has a book[4] on functional programming in javascript, which I own but haven't gotten around to reading yet, so I can't vouch for it. However, it does seem highly applicable.
Huge and seemingly often unacknowledged issue these days. And many attempted solutions seem to be adding fuel to the fire (or salt to the wound) by creating more tools (to fix problems with previous tools) ...
Red (red-lang.org) is one different sort of attempt at tackling modern software development complexity. Its like an improved version of REBOL, but aims to be a lot more - like a single language (two actually, RED/System and RED itself) for writing (close to) the full stack of development work, from low level to high level. Early days though, and they might not have anything much for web dev yet (though REBOL had/has plenty of Internet protocol and handling support).
If you are counting on people slowing down, you're in the wrong business.
You're asking the wrong question. It shouldn't be "how do we get people to slow down?" It should be, "how do we make rapid software development better?"
It's probably just a matter of time. Because all software ecosystems goes through its phases of "maturity" regarding testing.
Not too long ago (in human-years, not internet-years). Most node packages weren't built with unit testing. Now its quite common in the popular packages.
Website UI is probably the same thing. After all, it took us a really long time till we got the whole HTML5 spec finally stabilised.
So you will probably see the tipping point occur over the next 10 human years, or less.
And just like you I been really frustrated with the inadequacy of UI testing tools, especially with Selenium. So like @donaltroddyn, I set out to develop my own UI testing tool (https://uilicious.com/), to simplify the test flow and experience.
So wait around, you will see new tools, and watch them learn from one another. And if you want to dive right into it, we are currently running close beta.
Then, considering that if people don't spend sufficient time/money on proper testing, how much time/money do they spend on security? Probably even less.
As an example, one survey (https://ashleynolan.co.uk/blog/frontend-tooling-survey-2016-...) put the number of developers who don't use any test tools at almost 50%. In the same survey about 80% of people stated their level of JS knowledge was Intermediate, Advanced or Expert.