Introducing Aptly that Turns Ideas into Working Apps

Mar 21, 2022 hal's Blog

Speak Your Mind: Introducing Aptly, the Software Platform that Turns Ideas into Working Apps

Evan Patton, Ashley Granquist, Maura Kelleher, Hal Abelson and the MIT App Inventor Team

What if you could create apps simply by describing what you want in everyday language? How might you use that power to improve life for your family, your community, the world? What would you build?

MIT App Inventor is dedicated to the vision that anyone should be able to create apps, even if they have no background in coding or technology. App Inventor’s drag-and-drop, blocks-based programming has been a key enabler here for more than a decade, with almost 15 million users who among them have created more than 65 million apps.

The MIT App Inventor team is taking a step further in realizing this vision through its research on Aptly, a new tool that transforms verbal descriptions into working apps. For example, if you tell the computer in English —

Make an app with a text box, a list of six languages and a button that says “translate.” When the button is clicked, translate the text into the selected language and show the translation.

The result is a working app that does exactly this, as shown below running on a phone. It’s like expressing an idea for an app you want and instantly having it on your phone.

In generating the app, Aptly goes beyond the literal words and phrases in the English description. In this example, Aptly has independently chosen the six languages to be English, Spanish, French, German, Italian, and Japanese, and it has inferred that there should be a box where the user can provide the text to be translated.

In contrast, even the simplest coding platforms require technical work. Users of present-day App Inventor would create the translation app by specifying components and their properties as shown here:

and assembling blocks as shown here:

With Aptly, the system automatically creates these components, properties, and blocks, then passes them to the rest of App Inventor, which generates the working app. Once you have the text and the blocks, you can modify and edit them to extend the application.

The following video shows a similar example using Aptly, where the user speaks rather than types the description of the desired app:

How Aptly works

Aptly is one of an emerging family of “large language” development platforms that generate text from natural language input, including generating computer code. These models are built by training on huge numbers of examples of the kinds of text they should process. Examples of these large language models are OpenAI’s GPT-3 and Google’s LaMDA. Aptly uses OpenAI Codex, an offshoot of GPT-3 trained to generate code in Python, HTML, and other languages. Aptly builds upon Codex with additional training for code in a language similar to Python that has been specialized to have a one-to-one correspondence with App Inventor’s blocks.

Here’s the textual language Aptly generated in creating the translation app. Aptly generated this text automatically using the English text input shown above. Once it creates the text, Aptly parses it to produce App Inventor blocks and apps:

Screen1 = Screen()
TextBox1 = TextBox(Screen1, Hint = "Enter text here")
LanguageList = ListView(Screen1, ElementsFromString = "English,Spanish,French,German,Italian,Japanese")
TranslateButton = Button(Screen1, Text = "Translate")
TranslationLabel = Label(Screen1)
Translator1 = Translator(Screen1)

when TranslateButton.Click():
call Translator1.RequestTranslation (lists_select_item (["en", "es", "fr", "de", "it", "ja"], LanguageList.Selection), TextBox1.Text)

when Translator1.GotTranslation(responseCode, translation):
set TranslationLabel.Text = translation

Aptly’s text language design was motivated by the notion that close proximity to the Python syntax would allow the platform to take advantage of Codex’s Python training for basic language structures (e.g., that procedure arguments should be separated by commas and enclosed in balanced parentheses) while also maintaining a correspondence with App Inventor blocks.

Aptly’s challenge to computational thinking education

The role of programming (“coding”) in computational thinking remains a topic of active discussion among computer education curriculum developers. Some developers regard practice with coding as fundamental to the curriculum. Others consider coding to be marginal to computational thinking, preferring to focus on conceptual ideas in computation, independent of how they are expressed in modern computer languages.

Aptly aligns with the second view. It’s closely related to today’s burgeoning “no-code” movement that seeks to lessen the need for expert programmers in building industrial applications. No-code aims to make software development accessible to everyone, removing barriers so that people without extensive training in coding can bring ideas to life. Aptly offers the same approach in K-12 education, where software tools could eliminate much of the need for learners to deal with code.

Much of today’s activity in computational thinking traces back to the 1970’s MIT work of Seymour Papert, where a key principle was that children can practice formal thinking by expressing ideas in a specially designed computer language (Logo). What becomes of this approach when Aptly and its successors let the computer automatically recast ordinary language as code? Is the computer now doing the computational thinking on its own? Has the need to teach computational thinking been obviated?

Aptly points to a new perspective on computational thinking and raises new questions: Does the tool diminish the need for humans to engage in computational thinking, and does it signal an end to some of the core skills associated with computational thinking? Or will it inspire more people to dig deeper into the process of creating apps and let them address new computing possibilities that transcend coding? Or could it represent a combination of both?

More research in learning outcomes both with beginners and experienced coders is needed. There are new educational possibilities to explore. For example, learners with some experience can get started on an app using Aptly and then edit the Aptly-created blocks to add features themselves. People who want to practice debugging can describe an intended app, have Aptly create one that mostly matches the description, and then practice finding any bugs that cause the Aptly-generated app to not function as intended.

In addition, Aptly may provide evidence of surprising features of contemporary machine learning as it applies to computer science tasks, such as the idea that computers can generalize beyond the specific instructions they have been given. For example, in creating the translation app described above, Aptly independently chose the six languages to target based on what it had encountered in its training.

Aptly’s future

Aptly is currently a limited prototype. It does not support every App Inventor feature and its Codex-based training uses only a few examples. Nevertheless, today’s explosive activity in large language models and no-code platforms gives confidence that the performance of Aptly and similar tools will improve and that their use in K-12 education will grow.

What will be the impact of tools like Aptly in computing education? Consider an analogy with the use of calculators in mathematics education. The introduction of calculators in the 1970s sparked debates about their use in education that are still ongoing after 50 years. Some educators resist calculators, worrying that they would become crutches that undermine the learning of arithmetic skills. Others embrace the opportunity for calculators to be springboards that let students move beyond arithmetic to access higher-level mathematics concepts.

Will Aptly be a crutch or a springboard?

We can expect similar debates in computational thinking education around coding skills. This is our vision for Aptly: Children and adults alike will be able to create meaningful apps without prior education in programming. Creating apps — once inaccessible to non-programmers — will require less expertise, allowing people’s ideas to blossom. That’s both a challenge and an opportunity for practitioners of educational computing.