10f66f451Sopenharmony_ci<html><head><title>toybox FAQ</title> 20f66f451Sopenharmony_ci<!--#include file="header.html" --> 30f66f451Sopenharmony_ci 40f66f451Sopenharmony_ci<h1>Frequently Asked Questions</h1> 50f66f451Sopenharmony_ci 60f66f451Sopenharmony_ci<h2>General Questions</h2> 70f66f451Sopenharmony_ci 80f66f451Sopenharmony_ci<ul> 90f66f451Sopenharmony_ci<li><h2><a href="#why_toybox">Why toybox? (What was wrong with busybox?)</a></h2></li> 100f66f451Sopenharmony_ci<li><h2><a href="#capitalize">Do you capitalize toybox?</a></h2></li> 110f66f451Sopenharmony_ci<li><h2><a href="#support_horizon">Why a 7 year support horizon?</a></h2></li> 120f66f451Sopenharmony_ci<li><h2><a href="#releases">Why time based releases?</a></h2></li> 130f66f451Sopenharmony_ci<li><h2><a href="#code">Where do I start understanding the toybox source code?</a></h2></li> 140f66f451Sopenharmony_ci<li><h2><a href="#when">When were historical toybox versions released?</a></h2></li> 150f66f451Sopenharmony_ci<li><h2><a href="#bugs">Where do I report bugs?</a></h2></li> 160f66f451Sopenharmony_ci<li><h2><a href="#b_links">What are those /b/number links in the git log?</a></h2></li> 170f66f451Sopenharmony_ci<li><h2><a href="#opensource">What is the relationship between toybox and android?</a></h2></li> 180f66f451Sopenharmony_ci<li><h2><a href="#backporting">Will you backport fixes to old versions?</a></h2></li> 190f66f451Sopenharmony_ci<li><h2><a href="#dotslash">What's this ./ on the front of commands in your examples?</a></h2></li> 200f66f451Sopenharmony_ci 210f66f451Sopenharmony_ci</ul> 220f66f451Sopenharmony_ci 230f66f451Sopenharmony_ci<h2>Using toybox</h2> 240f66f451Sopenharmony_ci 250f66f451Sopenharmony_ci<ul> 260f66f451Sopenharmony_ci<!-- get binaries --> 270f66f451Sopenharmony_ci<li><h2><a href="#install">How do I install toybox?</h2></li> 280f66f451Sopenharmony_ci<li><h2><a href="#cross">How do I cross compile toybox?</h2></li> 290f66f451Sopenharmony_ci<li><h2><a href="#system">What part of Linux/Android does toybox provide?</h2></li> 300f66f451Sopenharmony_ci<li><h2><a href="#mkroot">How do I build a working Linux system with toybox?</a></h2></li> 310f66f451Sopenharmony_ci</ul> 320f66f451Sopenharmony_ci 330f66f451Sopenharmony_ci<hr /><h2><a name="why_toybox" />Q: "Why is there toybox? What was wrong with busybox?"</h2> 340f66f451Sopenharmony_ci 350f66f451Sopenharmony_ci<p>A: Toybox started back in 2006 when I (Rob Landley) 360f66f451Sopenharmony_ci<a href=https://lwn.net/Articles/202106/>handed off BusyBox maintainership</a> 370f66f451Sopenharmony_ciand <a href=http://landley.net/notes-2006.html#28-09-2006>started over from 380f66f451Sopenharmony_ciscratch</a> on a new codebase after a 390f66f451Sopenharmony_ci<a href=http://lists.busybox.net/pipermail/busybox/2006-September/058617.html>protracted licensing argument</a> took all the fun out of working on BusyBox.</p> 400f66f451Sopenharmony_ci 410f66f451Sopenharmony_ci<p>Toybox was just a personal project until it got 420f66f451Sopenharmony_ci<a href=http://landley.net/notes-2011.html#13-11-2011>relaunched</a> 430f66f451Sopenharmony_ciin November 2011 with a new goal to make Android 440f66f451Sopenharmony_ci<a href=http://landley.net/aboriginal/about.html#selfhost>self-hosting</a>. 450f66f451Sopenharmony_ciThis involved me relicensing my own 460f66f451Sopenharmony_cicode, which made people who had never used or participated in the project 470f66f451Sopenharmony_ci<a href=https://lwn.net/Articles/478308/>loudly angry</a>. The switch came 480f66f451Sopenharmony_ciafter a lot of thinking <a href=http://landley.net/talks/ohio-2013.txt>about 490f66f451Sopenharmony_cilicenses</a> and <a href=http://landley.net/notes-2011.html#21-03-2011>the 500f66f451Sopenharmony_citransition to smartphones</a>, which led to a 510f66f451Sopenharmony_ci<a href=https://www.youtube.com/watch?v=SGmtP5Lg_t0>2013 talk</a> laying 520f66f451Sopenharmony_ciout a 530f66f451Sopenharmony_ci<a href=http://landley.net/talks/celf-2013.txt>strategy</a> 540f66f451Sopenharmony_cito make Android self-hosting using toybox. This helped 550f66f451Sopenharmony_ci<a href=https://code.google.com/p/android/issues/detail?id=76861>bring 560f66f451Sopenharmony_ciit to Android's attention</a>, and they 570f66f451Sopenharmony_ci<a href=https://lwn.net/Articles/629362/>merged it</a> into Android M.</p> 580f66f451Sopenharmony_ci 590f66f451Sopenharmony_ci<p>The unfixable problem with busybox was licensing: BusyBox predates Android 600f66f451Sopenharmony_ciby almost a decade, but Android still doesn't ship with it because GPLv3 came 610f66f451Sopenharmony_ciout around the same time Android did and caused many people to throw 620f66f451Sopenharmony_ciout the GPLv2 baby with the GPLv3 bathwater. 630f66f451Sopenharmony_ciAndroid <a href=https://source.android.com/source/licenses.html>explicitly 640f66f451Sopenharmony_cidiscourages</a> use of GPL and LGPL licenses in its products, and has gradually 650f66f451Sopenharmony_cireimplemented historical GPL components (such as its bluetooth stack) under the 660f66f451Sopenharmony_ciApache license. Apple's 670f66f451Sopenharmony_ci<a href=http://meta.ath0.com/2012/02/05/apples-great-gpl-purge/>less subtle</a> response was to freeze xcode at the last GPLv2 releases 680f66f451Sopenharmony_ci(GCC 4.2.1 with binutils 2.17) for over 5 years while sponsoring the 690f66f451Sopenharmony_cidevelopment of new projects (clang/llvm/lld) to replace them, 700f66f451Sopenharmony_ciimplementing a 710f66f451Sopenharmony_ci<a href=https://www.osnews.com/story/24572/apple-ditches-samba-in-favour-of-homegrown-replacement/>new SMB server</a> from scratch to 720f66f451Sopenharmony_ci<a href=https://archive.org/details/copyleftconf2020-allison>replace samba</a>, 730f66f451Sopenharmony_ciswitching <a href=https://www.theverge.com/2019/6/4/18651872/apple-macos-catalina-zsh-bash-shell-replacement-features>bash with zsh</a>, and so on. 740f66f451Sopenharmony_ciToybox itself exists because somebody with in a legacy position 750f66f451Sopenharmony_cijust wouldn't shut up about GPLv3, otherwise I would probably 760f66f451Sopenharmony_cistill happily be maintaining BusyBox. (For more on how I wound 770f66f451Sopenharmony_ciup working on busybox in the first place, 780f66f451Sopenharmony_ci<a href=http://landley.net/aboriginal/history.html>see here</a>.)</p> 790f66f451Sopenharmony_ci 800f66f451Sopenharmony_ci<hr /><h2><a name="capitalize" />Q: Do you capitalize toybox?</h2> 810f66f451Sopenharmony_ci 820f66f451Sopenharmony_ci<p>A: Only at the start of a sentence. The command name is all lower case so 830f66f451Sopenharmony_ciit seems silly to capitalize the project name, but not capitalizing the 840f66f451Sopenharmony_cistart of sentences is awkward, so... compromise. (It is _not_ "ToyBox".)</p> 850f66f451Sopenharmony_ci 860f66f451Sopenharmony_ci<hr /><h2><a name="support_horizon">Q: Why a 7 year support horizon?</a></h2> 870f66f451Sopenharmony_ci 880f66f451Sopenharmony_ci<p>A: Our <a href=http://lists.busybox.net/pipermail/busybox/2006-September/058440.html>longstanding rule of thumb</a> is to try to run and build on 890f66f451Sopenharmony_cihardware and distributions released up to 7 years ago, and feel ok dropping 900f66f451Sopenharmony_cisupport for stuff older than that. (This is a little longer than Ubuntu's 910f66f451Sopenharmony_ciLong Term Support, but not by much.)</p> 920f66f451Sopenharmony_ci 930f66f451Sopenharmony_ci<p>My original theory was "4 to 5 of the 18-month cycles of moore's law should cover 940f66f451Sopenharmony_cithe vast majority of the installed base of PC hardware", loosely based on some 950f66f451Sopenharmony_ciresearch I did <a href=http://www.catb.org/esr/halloween/halloween9.html#id2867629>back in 2003</a> 960f66f451Sopenharmony_ciand <a href=http://catb.org/esr/writings/world-domination/world-domination-201.html#id248066>updated in 2006</a> 970f66f451Sopenharmony_ciwhich said that low end systems were 2 iterations of moore's 980f66f451Sopenharmony_cilaw below the high end systems, and that another 2-3 iterations should cover 990f66f451Sopenharmony_cithe useful lifetime of most systems no longer being sold but still in use and 1000f66f451Sopenharmony_cipotentially being upgraded to new software releases.</p> 1010f66f451Sopenharmony_ci 1020f66f451Sopenharmony_ci<p>That analysis missed <a href=http://landley.net/notes-2011.html#26-06-2011>industry 1030f66f451Sopenharmony_cichanges</a> in the 1990's that stretched the gap 1040f66f451Sopenharmony_cifrom low end to high end from 2 cycles to 4 cycles, and ignored 1050f66f451Sopenharmony_ci<a href=https://landley.net/notes-2010.html#09-10-2010>the switch</a> from PC to smartphone cutting off the R&D air supply of the 1060f66f451Sopenharmony_cilaptop market. Meanwhile the Moore's Law <a href=https://en.wikipedia.org/wiki/Logistic_function>s-curve</a> started bending back down (as they 1070f66f451Sopenharmony_ci<a href=https://en.wikipedia.org/wiki/Diffusion_of_innovations>always do</a>) 1080f66f451Sopenharmony_ciback in 2000, and these days is pretty flat: the drive for faster clock 1090f66f451Sopenharmony_cispeeds <a href=http://www.anandtech.com/show/613>stumbled</a> 1100f66f451Sopenharmony_ciand <a href=http://www.pcworld.com/article/118603/article.html>died</a>, with 1110f66f451Sopenharmony_cithe subsequent drive to go "wide" maxing out for most applications 1120f66f451Sopenharmony_ciaround 4x SMP with maybe 2 megabyte caches. These days the switch from exponential to 1130f66f451Sopenharmony_cilinear growth in hardware capabilities is 1140f66f451Sopenharmony_ci<a href=https://www.cnet.com/news/end-of-moores-law-its-not-just-about-physics/>common knowledge</a> and 1150f66f451Sopenharmony_ci<a href=http://www.acm.org/articles/people-of-acm/2016/david-patterson>widely 1160f66f451Sopenharmony_ciaccepted</a>.</p> 1170f66f451Sopenharmony_ci 1180f66f451Sopenharmony_ci<p>But the 7 year rule of thumb stuck around anyway: if a kernel or libc 1190f66f451Sopenharmony_cifeature is less than 7 years old, I try to have a build-time configure test 1200f66f451Sopenharmony_cifor it to let the functionality cleanly drop out. I also keep old Ubuntu 1210f66f451Sopenharmony_ciimages around in VMs to perform the occasional defconfig build there to 1220f66f451Sopenharmony_cisee what breaks. (I'm not perfect about this, but I accept bug reports.)</p> 1230f66f451Sopenharmony_ci 1240f66f451Sopenharmony_ci<hr /><h2><a name="releases" />Q: Why time based releases?</h2> 1250f66f451Sopenharmony_ci<p>A: Toybox targets quarterly releases (a similar schedule to the Linux 1260f66f451Sopenharmony_cikernel) because Martin Michlmayr's excellent 1270f66f451Sopenharmony_ci<a href=http://www.youtube.com/watch?v=IKsQsxubuAA>talk on the 1280f66f451Sopenharmony_cisubject</a> was convincing. This is actually two questions, "why have 1290f66f451Sopenharmony_cireleases" and "why schedule them".</p> 1300f66f451Sopenharmony_ci 1310f66f451Sopenharmony_ci<p>Releases provide synchronization points where the developers certify 1320f66f451Sopenharmony_ci"it worked for me". Each release is a known version with predictable behavior, 1330f66f451Sopenharmony_ciand right or wrong at least everyone should be seeing 1340f66f451Sopenharmony_cisimilar results so might be able to google an unexpected outcome. 1350f66f451Sopenharmony_ciReleases focus end-user testing on specific versions 1360f66f451Sopenharmony_ciwhere issues can be reproduced, diagnosed, and fixed. 1370f66f451Sopenharmony_ciReleases also force the developers to do periodic tidying, packaging, 1380f66f451Sopenharmony_cidocumentation review, finish up partially implemented features languishing 1390f66f451Sopenharmony_ciin their private trees, and give regular checkpoints to measure progress.</p> 1400f66f451Sopenharmony_ci 1410f66f451Sopenharmony_ci<p>Changes accumulate over time: different feature sets, data formats, 1420f66f451Sopenharmony_cicontrol knobs... Toybox's switch from "ls -q" to "ls -b" as the default output 1430f66f451Sopenharmony_ciformat was not-a-bug-it's-a "design improvement", but the 1440f66f451Sopenharmony_cidifference is academic if the change breaks somebody's script. 1450f66f451Sopenharmony_ciReleases give you the option to schedule upgrades as maintenance, not to rock 1460f66f451Sopenharmony_cithe boat just now, and use a known working release version until later.</p> 1470f66f451Sopenharmony_ci 1480f66f451Sopenharmony_ci<p>The counter-argument is that "continuous integration" 1490f66f451Sopenharmony_cican be made robust with sufficient automated testing. But like the 1500f66f451Sopenharmony_ci<a href=https://web.archive.org/web/20131123071427/http://www.shirky.com/weblog/2013/11/healthcare-gov-and-the-gulf-between-planning-and-reality/>waterfall method</a>, this places insufficent 1510f66f451Sopenharmony_ciemphasis on end-user feedback and learning from real world experience. 1520f66f451Sopenharmony_ciDeveloper testing is either testing that the code does what the developers 1530f66f451Sopenharmony_ciexpect given known inputs running in an established environment, or it's 1540f66f451Sopenharmony_ciregression testing against bugs previously found in the field. No plan 1550f66f451Sopenharmony_cisurvives contact with the enemy, and technology always breaks once it 1560f66f451Sopenharmony_cileaves the lab and encounters real world data and use cases in new 1570f66f451Sopenharmony_ciruntime and build environments.</p> 1580f66f451Sopenharmony_ci 1590f66f451Sopenharmony_ci<p>The best way to give new users a reasonable first experience is to point 1600f66f451Sopenharmony_cithem at specific stable versions where development quiesced and 1610f66f451Sopenharmony_ciextra testing occurred. There will still be teething troubles, but multiple 1620f66f451Sopenharmony_cipeople experiencing the _same_ teething troubles can potentially 1630f66f451Sopenharmony_cihelp each other out.</p> 1640f66f451Sopenharmony_ci 1650f66f451Sopenharmony_ci<p>Releases on a schedule are better than releases "when it's ready" for 1660f66f451Sopenharmony_cithe same reason a regularly scheduled bus beats one that leaves when it's 1670f66f451Sopenharmony_ci"full enough": the schedule lets its users make plans. Even if the bus leaves 1680f66f451Sopenharmony_ciempty you know when the next one arrives so missing this one isn't a disaster. 1690f66f451Sopenharmony_ciand starting the engine to leave doesn't provoke a last-minute rush of nearby 1700f66f451Sopenharmony_cinot-quite-ready passengers racing to catch it causing further delay and 1710f66f451Sopenharmony_cirepeated start/stop cycles as it ALMOST leaves. 1720f66f451Sopenharmony_ci(The video in the first paragraph goes into much greater detail.)</p> 1730f66f451Sopenharmony_ci 1740f66f451Sopenharmony_ci<hr /><h2><a name="code" />Q: Where do I start understanding the source code?</h2> 1750f66f451Sopenharmony_ci 1760f66f451Sopenharmony_ci<p>A: Toybox is written in C. There are longer writeups of the 1770f66f451Sopenharmony_ci<a href=design.html>design ideas</a> and a <a href=code.html>code walkthrough</a>, 1780f66f451Sopenharmony_ciand the <a href=about.html>about page</a> summarizes what we're trying to 1790f66f451Sopenharmony_ciaccomplish, but here's a quick start:</p> 1800f66f451Sopenharmony_ci 1810f66f451Sopenharmony_ci<p>Toybox uses the standard three stage configure/make/install 1820f66f451Sopenharmony_ci<a href=code.html#building>build</a>, in this case "<b>make defconfig; 1830f66f451Sopenharmony_cimake; make install</b>". Type "<b>make help</b>" to 1840f66f451Sopenharmony_cisee available make targets.</p> 1850f66f451Sopenharmony_ci 1860f66f451Sopenharmony_ci<p><u>The configure stage</u> is copied from the Linux kernel (in the "kconfig" 1870f66f451Sopenharmony_cidirectory), and saves your selections in the file ".config" at the top 1880f66f451Sopenharmony_cilevel. The "<b>make defconfig</b>" target selects the 1890f66f451Sopenharmony_cimaximum sane configuration (enabling all the commands and features that 1900f66f451Sopenharmony_ciaren't unfinished, or only intended as examples, or debug code...) and is 1910f66f451Sopenharmony_ciprobably what you want. You can use "<b>make menuconfig</b>" to manually select 1920f66f451Sopenharmony_cispecific commands to include, through an interactive menu (cursor up and 1930f66f451Sopenharmony_cidown, enter to descend into a sub-menu, space to select an entry, ? to see 1940f66f451Sopenharmony_cian entry's help text, esc to exit). The menuconfig help text is the 1950f66f451Sopenharmony_cisame as the command's "<b>--help</b>" output.</p> 1960f66f451Sopenharmony_ci 1970f66f451Sopenharmony_ci<p><u>The "make" stage</u> creates a toybox binary (which is stripped, look in 1980f66f451Sopenharmony_cigenerated/unstripped for the debug versions), and "<b>make install</b>" adds a bunch of 1990f66f451Sopenharmony_cisymlinks to toybox under the various command names. Toybox determines which 2000f66f451Sopenharmony_cicommand to run based on the filename, or you can use the "toybox" name in which case the first 2010f66f451Sopenharmony_ciargument is the command to run (ala "toybox ls -l").</p> 2020f66f451Sopenharmony_ci 2030f66f451Sopenharmony_ci<p><u>You can also build 2040f66f451Sopenharmony_ciindividual commands as standalone executables</u>, ala "make sed cat ls". 2050f66f451Sopenharmony_ciThe "make change" target builds all of them, as in "change for a $20".</p> 2060f66f451Sopenharmony_ci 2070f66f451Sopenharmony_ci<p><u>The main() function is in main.c</u> at the top level, 2080f66f451Sopenharmony_cialong with setup plumbing and selecting which command to run this time. 2090f66f451Sopenharmony_ciThe function toybox_main() in the same file implements the "toybox" 2100f66f451Sopenharmony_cimultiplexer command that lists and selects the other commands.</p> 2110f66f451Sopenharmony_ci 2120f66f451Sopenharmony_ci<p><u>The individual command implementations are under "toys"</u>, and are grouped 2130f66f451Sopenharmony_ciinto categories (mostly based on which standard they come from, posix, lsb, 2140f66f451Sopenharmony_ciandroid...) The "pending" directory contains unfinished commands, and the 2150f66f451Sopenharmony_ci"examples" directory contains example code that aren't really useful commands. 2160f66f451Sopenharmony_ciCommands in those two directories 2170f66f451Sopenharmony_ciare _not_ selected by defconfig. (Most of the files in the pending directory 2180f66f451Sopenharmony_ciare third party submissions that have not yet undergone 2190f66f451Sopenharmony_ci<a href=cleanup.html>proper code review</a>.)</p> 2200f66f451Sopenharmony_ci 2210f66f451Sopenharmony_ci<p><u>Common infrastructure shared between commands is under "lib"</u>. Most 2220f66f451Sopenharmony_cicommands call lib/args.c to parse their command line arguments before calling 2230f66f451Sopenharmony_cithe command's own main() function, which uses the option string in 2240f66f451Sopenharmony_cithe command's NEWTOY() macro. This is similar to the libc function getopt(), 2250f66f451Sopenharmony_cibut more powerful, and is documented at the top of lib/args.c. A NULL option 2260f66f451Sopenharmony_cistring prevents this code from being called for that command.</p> 2270f66f451Sopenharmony_ci 2280f66f451Sopenharmony_ci<p><u>The build/install infrastructure is shell scripts under 2290f66f451Sopenharmony_ci"scripts"</u> (starting with scripts/make.sh and scripts/install.sh). 2300f66f451Sopenharmony_ci<u>These populate the "generated" directory</u> with headers 2310f66f451Sopenharmony_cicreated from other files, which are <a href=code.html#generated>described</a> 2320f66f451Sopenharmony_ciin the code walkthrough. All the 2330f66f451Sopenharmony_cibuild's temporary files live under generated, including the .o files built 2340f66f451Sopenharmony_cifrom the .c files (in generated/obj). The "make clean" target deletes that 2350f66f451Sopenharmony_cidirectory. ("make distclean" also deletes your .config and deletes the 2360f66f451Sopenharmony_cikconfig binaries that process .config.)</p> 2370f66f451Sopenharmony_ci 2380f66f451Sopenharmony_ci<p><u>Each command's .c file contains all the information for that command</u>, so 2390f66f451Sopenharmony_ciadding a command to toybox means adding a single file under "toys". 2400f66f451Sopenharmony_ciUsually you <a href=code.html#adding>start a new command</a> by copying an 2410f66f451Sopenharmony_ciexisting command file to a new filename 2420f66f451Sopenharmony_ci(toys/examples/hello.c, toys/examples/skeleton.c, toys/posix/cat.c, 2430f66f451Sopenharmony_ciand toys/posix/true.c have all been used for this purpose) and then replacing 2440f66f451Sopenharmony_ciall instances of its old name with the new name (which should match the 2450f66f451Sopenharmony_cinew filename), and modifying the help text, argument string, and what the 2460f66f451Sopenharmony_cicode does. You might have to "make distclean" before your new command 2470f66f451Sopenharmony_cishows up in defconfig or menuconfig.</p> 2480f66f451Sopenharmony_ci 2490f66f451Sopenharmony_ci<p><u>The toybox test suite lives in the "tests" directory</u>, and is 2500f66f451Sopenharmony_cidriven by scripts/test.sh and scripts/runtest.sh. From the top 2510f66f451Sopenharmony_cilevel you can "make tests" to test everything, or "make test_sed" to test a 2520f66f451Sopenharmony_cisingle command's standalone version (which should behave identically, 2530f66f451Sopenharmony_cibut that's why we test). You can set TEST_HOST=1 to test the host version 2540f66f451Sopenharmony_ciinstead of the toybox version (in theory they should work the same), 2550f66f451Sopenharmony_ciand VERBOSE=all to see diffs of the expected and actual output for all 2560f66f451Sopenharmony_cifailing tests. The default VERBOSE=fail stops at the first such failure.</p> 2570f66f451Sopenharmony_ci 2580f66f451Sopenharmony_ci<hr /><h2><a name="when" />Q: When were historical toybox versions released?</h2> 2590f66f451Sopenharmony_ci 2600f66f451Sopenharmony_ci<p>A: For vanilla releases, check the 2610f66f451Sopenharmony_ci<a href=https://github.com/landley/toybox/tags>date on the commit tag</a> 2620f66f451Sopenharmony_cior <a href=https://landley.net/toybox/downloads/binaries/>the 2630f66f451Sopenharmony_ciexample binaries</a> against the output of "toybox --version". 2640f66f451Sopenharmony_ciBetween releases the --version 2650f66f451Sopenharmony_ciinformation is in "git describe --tags" format with "tag-count-hash" showing the 2660f66f451Sopenharmony_cimost recent commit tag, the number of commits since that tag, and 2670f66f451Sopenharmony_cithe hash of the current commit.</p> 2680f66f451Sopenharmony_ci 2690f66f451Sopenharmony_ci<p>Android makes its own releases on its own 2700f66f451Sopenharmony_ci<a href=https://en.wikipedia.org/wiki/Android_version_history>schedule</a> 2710f66f451Sopenharmony_ciusing its own version tags, but lists corresponding upstream toybox release 2720f66f451Sopenharmony_civersions <a href=https://android.googlesource.com/platform/system/core/+/master/shell_and_utilities/README.md>here</a>. For more detail you can look up 2730f66f451Sopenharmony_ci<a href=https://android.googlesource.com/platform/external/toybox/+refs>AOSP's 2740f66f451Sopenharmony_cigit tags</a>. (The <a href=https://source.android.com/setup/start>Android Open Source Project</a> is the "upstream" android vendors 2750f66f451Sopenharmony_cistart form when making their own releases. Google's phones run AOSP versions 2760f66f451Sopenharmony_civerbatim, other vendors tend to take those releases as starting points to 2770f66f451Sopenharmony_cimodify.)</p> 2780f66f451Sopenharmony_ci 2790f66f451Sopenharmony_ci<p>If you want to find the vanilla toybox commit corresponding to an AOSP 2800f66f451Sopenharmony_citoybox version, find the most recent commit in the android log that isn't from a 2810f66f451Sopenharmony_ci@google or @android address and search for it in the vanilla commit log. 2820f66f451Sopenharmony_ci(The timestamp should match but the hash will differ, 2830f66f451Sopenharmony_cibecause each git hash includes the previous 2840f66f451Sopenharmony_cigit hash in the data used to generate it so all later commits have a different 2850f66f451Sopenharmony_cihash if any of the tree's history differs; yes Linus Torvalds published 3 years 2860f66f451Sopenharmony_cibefore Satoshi Nakamoto.) Once you've identified the vanilla commit's hash, 2870f66f451Sopenharmony_ci"git describe --tags $HASH" in the vanilla tree should give you the --version 2880f66f451Sopenharmony_ciinfo for that one.</p> 2890f66f451Sopenharmony_ci 2900f66f451Sopenharmony_ci<hr /><h2><a name="bugs" />Q: Where do I report bugs?</h2> 2910f66f451Sopenharmony_ci 2920f66f451Sopenharmony_ci<p>A: Ideally on the <a href=http://lists.landley.net/listinfo.cgi/toybox-landley.net>mailing list</a>, although <a href=mailto:rob@landley.net>emailing the 2930f66f451Sopenharmony_cimaintainer</a> is a popular if slightly less reliable alternative. 2940f66f451Sopenharmony_ciIssues submitted to <a href=https://github.com/landley/toybox>github</a> 2950f66f451Sopenharmony_ciare generally dealt with less promptly, but mostly get done eventually. 2960f66f451Sopenharmony_ciAOSP has its <a href=https://source.android.com/setup/contribute/report-bugs>own bug reporting mechanism</a> (although for toybox they usually forward them 2970f66f451Sopenharmony_cito the mailing list) and Android vendors usually forward them to AOSP which 2980f66f451Sopenharmony_ciforwards them to the list.</p> 2990f66f451Sopenharmony_ci 3000f66f451Sopenharmony_ci<p>Note that if we can't reproduce a bug, we probably can't fix it. 3010f66f451Sopenharmony_ciNot only does this mean providing enough information for us to see the 3020f66f451Sopenharmony_cibehavior ourselves, but ideally doing so in a reasonably current version. 3030f66f451Sopenharmony_ciThe older it is the greater the chance somebody else found and fixed it 3040f66f451Sopenharmony_cialready, so the more out of date the version you're reporting a bug against 3050f66f451Sopenharmony_cithe less effort we're going to put into reproducing the problem.</p> 3060f66f451Sopenharmony_ci 3070f66f451Sopenharmony_ci<hr /><h2><a name="b_links" />Q: What are those /b/number bug report 3080f66f451Sopenharmony_cilinks in the git log?</h2> 3090f66f451Sopenharmony_ci 3100f66f451Sopenharmony_ci<p>A: It's a Google thing. Replace /b/$NUMBER with 3110f66f451Sopenharmony_cihttps://issuetracker.google.com/$NUMBER to read it outside the googleplex.</p> 3120f66f451Sopenharmony_ci 3130f66f451Sopenharmony_ci<hr /><a name="opensource" /><h2>Q: What is the relationship between toybox and android?</h2> 3140f66f451Sopenharmony_ci 3150f66f451Sopenharmony_ci<p>A: The <a href=about.html>about page</a> tries to explain that, 3160f66f451Sopenharmony_ciand Linux Weekly News has covered toybox's history a 3170f66f451Sopenharmony_ci<a href=https://lwn.net/Articles/202106/>little</a> 3180f66f451Sopenharmony_ci<a href=https://lwn.net/Articles/478308/>over</a> 3190f66f451Sopenharmony_ci<a href=https://lwn.net/Articles/616272/>the</a> 3200f66f451Sopenharmony_ci<a href=https://lwn.net/Articles/629362/>years</a>.</p> 3210f66f451Sopenharmony_ci 3220f66f451Sopenharmony_ci<p>Toybox is a traditional open source project created and maintained 3230f66f451Sopenharmony_ciby hobbyist (volunteer) developers, originally for Linux but these days 3240f66f451Sopenharmony_cialso running on Android, BSD, and MacOS. The project started in 2006 3250f66f451Sopenharmony_ciand its original author (Rob Landley) 3260f66f451Sopenharmony_cicontinues to maintain the open source project.</p> 3270f66f451Sopenharmony_ci 3280f66f451Sopenharmony_ci<p>Android's base OS maintainer (Elliott Hughes, I.E. enh) 3290f66f451Sopenharmony_ci<a href=https://github.com/landley/toybox/commit/69a9f257234a>ported</a> 3300f66f451Sopenharmony_ci<a href=https://github.com/landley/toybox/commit/6a29bb1ebe62>toybox</a> 3310f66f451Sopenharmony_cito Android in 2014, merged it into Android M (Marshmallow), and remains 3320f66f451Sopenharmony_ciAndroid's toybox maintainer. (He explained it in his own words in 3330f66f451Sopenharmony_ci<a href=http://androidbackstage.blogspot.com/2016/07/episode-53-adb-on-adb.html>this podcast</a>, starting either 18 or 20 minutes in depending how 3340f66f451Sopenharmony_cimuch backstory you want.)</p> 3350f66f451Sopenharmony_ci 3360f66f451Sopenharmony_ci<p>Android's policy for toybox development is to push patches to the 3370f66f451Sopenharmony_ciopen source project (submitting them via the mailing list) then 3380f66f451Sopenharmony_ci"git pull" the public tree into Android's tree. To avoid merge conflicts, Android's 3390f66f451Sopenharmony_citree doesn't change any of the existing toybox files but instead adds <a href=https://android.googlesource.com/platform/external/toybox/+/refs/heads/master/Android.bp>parallel 3400f66f451Sopenharmony_cibuild infrastructure</a> off to one side. (Toybox uses a make wrapper around bash 3410f66f451Sopenharmony_ciscripts, AOSP builds with soong/ninja instead and checks in a snapshot of the 3420f66f451Sopenharmony_cigenerated/ directory to avoid running kconfig each build). 3430f66f451Sopenharmony_ciAndroid's changes to toybox going into the open source tree first 3440f66f451Sopenharmony_ciand being pulled from there into Android keeps the two trees in 3450f66f451Sopenharmony_cisync, and makes sure each change undergoes full open source design review 3460f66f451Sopenharmony_ciand discussion.</p> 3470f66f451Sopenharmony_ci 3480f66f451Sopenharmony_ci<p>Rob acknowledges Android is by far the largest userbase for the project, 3490f66f451Sopenharmony_cibut develops on a standard 64-bit Linux+glibc distro while building embedded 3500f66f451Sopenharmony_ci32-bit big-endian nommu musl systems requiring proper data alignment for work, 3510f66f451Sopenharmony_ciand is not a Google employee so does not have access 3520f66f451Sopenharmony_cito the Google build cluster of powerful machines capable of running the full 3530f66f451Sopenharmony_ciAOSP build in a reasonable amount of time. Rob is working to get android 3540f66f451Sopenharmony_cibuilding under android (the list of 3550f66f451Sopenharmony_citoybox tools Android's build uses is 3560f66f451Sopenharmony_ci<a href=https://android.googlesource.com/platform/prebuilts/build-tools/+/refs/heads/master/path/linux-x86/>here</a>, 3570f66f451Sopenharmony_ciand what else it needs from its build environment is 3580f66f451Sopenharmony_ci<a href=https://android.googlesource.com/platform/build/soong/+/refs/heads/master/ui/build/paths/config.go>here</a>), and he hopes someday to not only make a usable development 3590f66f451Sopenharmony_cienvironment out of it but also nudge the base OS towards a more granular 3600f66f451Sopenharmony_cipackage management system allowing you to upgrade things like toybox without 3610f66f451Sopenharmony_cia complete reinstall and reboot, plus the introduction of a "posix container" 3620f66f451Sopenharmony_ciwithin which you can not only run builds, but selinux lets you run binaries 3630f66f451Sopenharmony_ciyou've just built). In the meantime, Rob tests static bionic 3640f66f451Sopenharmony_cibuilds via the Android NDK when he remembers, but has limited time to work 3650f66f451Sopenharmony_cion toybox because it's not his day job. (The products his company makes ship 3660f66f451Sopenharmony_citoybox and they do sponsor the project's development, but it's one of many 3670f66f451Sopenharmony_ciresponsibilities at work.)</p> 3680f66f451Sopenharmony_ci 3690f66f451Sopenharmony_ci<p>Elliott is the Android base OS maintainer, in which role he manages 3700f66f451Sopenharmony_cia team of engineers. He also has limited time for toybox, both because it's one 3710f66f451Sopenharmony_ciof many packages he's responsible for (he maintains bionic, used to maintain 3720f66f451Sopenharmony_cidalvik...) and because he allowed himself to be promoted into management 3730f66f451Sopenharmony_ciand thus spends less time coding than he does sitting in meetings where testers 3740f66f451Sopenharmony_citalk to security people about vendor issues.</p> 3750f66f451Sopenharmony_ci 3760f66f451Sopenharmony_ci<p>Android has many other coders and security people who submit the occasional 3770f66f451Sopenharmony_citoybox patch, but of the last 1000 commits at the <a href=https://github.com/landley/toybox/commit/88b34c4bd3f8>time 3780f66f451Sopenharmony_ciof writing</a> this FAQ entry, Elliott submitted 276 and all other google.com 3790f66f451Sopenharmony_cior android.com addresses combined totaled 17. (Rob submitted 591, leaving 3800f66f451Sopenharmony_ci116 from other sources, but for both Rob and Elliott there's a lot of "somebody 3810f66f451Sopenharmony_cielse pointed out an issue, and then we wrote a patch". A lot of patches 3820f66f451Sopenharmony_cifrom both "Author:" lines thank someone else for the suggestion in the 3830f66f451Sopenharmony_cicommit comment.)</p> 3840f66f451Sopenharmony_ci 3850f66f451Sopenharmony_ci<hr /><a name="backporting" /><h2>Q: Will you backport fixes to old versions?</h2> 3860f66f451Sopenharmony_ci 3870f66f451Sopenharmony_ci<p>A: Probably not. The easiest thing to do is get your issue fixed upstream 3880f66f451Sopenharmony_ciin the current release, then get the newest version of the 3890f66f451Sopenharmony_ciproject built and running in the old environment.</p> 3900f66f451Sopenharmony_ci 3910f66f451Sopenharmony_ci<p>Backporting fixes generally isn't something open source projects run by 3920f66f451Sopenharmony_civolunteer developers do because the goal of the project's development community 3930f66f451Sopenharmony_ciis to extend and improve the project. We're happy to respond to our users' 3940f66f451Sopenharmony_cineeds, but if you're coming to the us for free tech support we're going 3950f66f451Sopenharmony_cito ask you to upgrade to a current version before we try to diagnose your 3960f66f451Sopenharmony_ciproblem.</p> 3970f66f451Sopenharmony_ci 3980f66f451Sopenharmony_ci<p>The volunteers are happy to fix any bugs you point out in the current 3990f66f451Sopenharmony_civersions because doing so helps everybody and makes the project better. We 4000f66f451Sopenharmony_ciwant to make the current version work for you. But diagnosing, debugging, and 4010f66f451Sopenharmony_cibackporting fixes to old versions doesn't help anybody but you, so isn't 4020f66f451Sopenharmony_cisomething we do for free. The cost of volunteer tech support is using a 4030f66f451Sopenharmony_cireasonably current version of the project.</p> 4040f66f451Sopenharmony_ci 4050f66f451Sopenharmony_ci<p>If you're using an old version built with an old 4060f66f451Sopenharmony_cicompiler on an old OS (kernel and libc), there's a fairly large chance 4070f66f451Sopenharmony_ciwhatever problem you're 4080f66f451Sopenharmony_ciseeing already got fixed, and to get that fix all you have to do is upgrade 4090f66f451Sopenharmony_cito a newer version. Diagnosing a problem that wasn't our bug means we spent 4100f66f451Sopenharmony_citime that only helps you, without improving the project. 4110f66f451Sopenharmony_ciIf you don't at least _try_ a current version, you're asking us for free 4120f66f451Sopenharmony_cipersonalized tech support.</p> 4130f66f451Sopenharmony_ci 4140f66f451Sopenharmony_ci<p>Reproducing bugs in current versions also makes our job easier. 4150f66f451Sopenharmony_ciThe further back in time 4160f66f451Sopenharmony_ciyou are, the more work it is for us digging back in the history to figure 4170f66f451Sopenharmony_ciout what we hadn't done yet in your version. If spot a problem in a git 4180f66f451Sopenharmony_cibuild pulled 3 days ago, it's obvious what changed and easy to fix or back out. 4190f66f451Sopenharmony_ciIf you ask about the current release version 3 months after it came out, 4200f66f451Sopenharmony_ciwe may have to think a while to remember what we did and there are a number of 4210f66f451Sopenharmony_cipossible culprits, but it's still tractable. If you ask about 3 year old 4220f66f451Sopenharmony_cicode, we have to reconstruct the history and the problem could be anything, 4230f66f451Sopenharmony_cithere's a lot more ground to cover and we haven't seen it in a while.</p> 4240f66f451Sopenharmony_ci 4250f66f451Sopenharmony_ci<p>As a rule of thumb, volunteers will generally answer polite questions about 4260f66f451Sopenharmony_cia given version for about three years after its release before it's so old 4270f66f451Sopenharmony_ciwe don't remember the answer off the top of our head. And if you want us to 4280f66f451Sopenharmony_ciput any _effort_ into tracking it down, we want you to put in a little effort 4290f66f451Sopenharmony_ciof your own by confirming it's still a problem with the current version 4300f66f451Sopenharmony_ci(I.E. we didn't fix it already). It's 4310f66f451Sopenharmony_cialso hard for us to fix a problem of yours if we can't reproduce it because 4320f66f451Sopenharmony_ciwe don't have any systems running an environment that old.</p> 4330f66f451Sopenharmony_ci 4340f66f451Sopenharmony_ci<p>If you don't want to upgrade, you have the complete source code and thus 4350f66f451Sopenharmony_cithe ability to fix it yourself, or can hire a consultant to do it for you. If 4360f66f451Sopenharmony_ciyou got your version from a vendor who still supports the older version, they 4370f66f451Sopenharmony_cican help you. But there are limits as to what volunteers will feel obliged to 4380f66f451Sopenharmony_cido for you.</p> 4390f66f451Sopenharmony_ci 4400f66f451Sopenharmony_ci<p>Commercial companies have different incentives. Your OS vendor, or 4410f66f451Sopenharmony_cihardware vendor for preinstalled systems, may have their own bug reporting 4420f66f451Sopenharmony_cimechanism and update channel providing backported fixes. And a paid consultant 4430f66f451Sopenharmony_ciwill happily set up a special environment just to reproduce your problem.</p> 4440f66f451Sopenharmony_ci 4450f66f451Sopenharmony_ci<hr /><h2><a name="install" />Q: How do I install toybox?</h2> 4460f66f451Sopenharmony_ci 4470f66f451Sopenharmony_ci<p>A: 4480f66f451Sopenharmony_ciMulticall binaries like toybox behave differently based on the filename 4490f66f451Sopenharmony_ciused to call them, so if you "mv toybox ls; ./ls -l" it acts like ls. Creating 4500f66f451Sopenharmony_cisymlinks or hardlinks and adding them to the $PATH lets you run the 4510f66f451Sopenharmony_cicommands normally by name, so that's probably what you want to do.</p> 4520f66f451Sopenharmony_ci 4530f66f451Sopenharmony_ci<p>If you already have a <a href=https://landley.net/toybox/downloads/binaries/>toybox binary</a> 4540f66f451Sopenharmony_ciyou can install a tree of command symlinks to 4550f66f451Sopenharmony_ci<a href=http://git.musl-libc.org/cgit/musl/tree/include/paths.h>the 4560f66f451Sopenharmony_cistandard path</a> 4570f66f451Sopenharmony_cilocations (<b>export PATH=/bin:/usr/bin:/sbin:/usr/sbin</b>) by doing:</p> 4580f66f451Sopenharmony_ci 4590f66f451Sopenharmony_ci<blockquote><p><b>for i in $(/bin/toybox --long); do ln -s /bin/toybox $i; done</b></p></blockquote> 4600f66f451Sopenharmony_ci 4610f66f451Sopenharmony_ci<p>Or you can install all the symlinks in the same directory as the toybox binary 4620f66f451Sopenharmony_ci(<b>export PATH="$PWD:$PATH"</b>) via:</p> 4630f66f451Sopenharmony_ci 4640f66f451Sopenharmony_ci<blockquote><p><b>for i in $(./toybox); do ln -s toybox $i; done</b></p></blockquote></p> 4650f66f451Sopenharmony_ci 4660f66f451Sopenharmony_ci<p>When building from source, use the "<b>make install</b>" and 4670f66f451Sopenharmony_ci"<b>make install_flat</b>" 4680f66f451Sopenharmony_citargets with an appropriate <b>PREFIX=/target/path</b> either 4690f66f451Sopenharmony_ciexported or on the make command line. When cross compiling, 4700f66f451Sopenharmony_ci"<b>make list</b>" outputs the command names enabled by defconfig. 4710f66f451Sopenharmony_ciFor more information, see "<b>make help</b>".</p> 4720f66f451Sopenharmony_ci 4730f66f451Sopenharmony_ci<p>The command name "toybox" takes the second argument as the name of the 4740f66f451Sopenharmony_cicommand to run, so "./toybox ls -l" also behaves like ls. The "toybox" 4750f66f451Sopenharmony_ciname is special in that it can have a suffix (toybox-i686 or toybox-1.2.3) 4760f66f451Sopenharmony_ciand still be recognized, so you can have multiple versions of toybox in the 4770f66f451Sopenharmony_cisame directory.</p> 4780f66f451Sopenharmony_ci 4790f66f451Sopenharmony_ci<p>When toybox doesn't recognize its 4800f66f451Sopenharmony_cifilename as a command, it dereferences one 4810f66f451Sopenharmony_cilevel of symlink. So if your script needs "gsed" you can "ln -s sed gsed", 4820f66f451Sopenharmony_cithen when you run "gsed" toybox knows how to be "sed".</p> 4830f66f451Sopenharmony_ci 4840f66f451Sopenharmony_ci<hr /><h2><a name="dotslash" />Q: What's this ./ on the front of commands in your examples?</h2> 4850f66f451Sopenharmony_ci 4860f66f451Sopenharmony_ci<p>A: When you don't give a path to a command's executable file, 4870f66f451Sopenharmony_cilinux command shells search the directories listed in the $PATH envionment 4880f66f451Sopenharmony_civariable (in order), which usually doesn't include the current directory 4890f66f451Sopenharmony_cifor security reasons. The 4900f66f451Sopenharmony_cimagic name "." indicates the current directory (the same way ".." means 4910f66f451Sopenharmony_cithe parent directory and starting with "/" means the root directory) 4920f66f451Sopenharmony_ciso "./file" gives a path to the executable file, and thus runs a command 4930f66f451Sopenharmony_ciout of the current directory where just typing "file" won't find it. 4940f66f451Sopenharmony_ciFor historical reasons PATH is colon-separated, and treats an 4950f66f451Sopenharmony_ciempty entry (including leading/trailing colon) as "check the current 4960f66f451Sopenharmony_cidirectory", so if you WANT to add the current directory to PATH you 4970f66f451Sopenharmony_cican PATH="$PATH:" but doing so is a TERRIBLE idea.</p> 4980f66f451Sopenharmony_ci 4990f66f451Sopenharmony_ci<p>Toybox's shell (toysh) checks for built-in commands before looking at the 5000f66f451Sopenharmony_ci$PATH (using the standard "bash builtin" logic just with lots more builtins), 5010f66f451Sopenharmony_ciso "ls" doesn't have to exist in your filesystem for toybox to find it. When 5020f66f451Sopenharmony_ciyou give a path to a command the shell won't run the built-in version 5030f66f451Sopenharmony_cibut will run the file at that location. (But the multiplexer command 5040f66f451Sopenharmony_ciwon't: "toybox /bin/ls" runs the built-in ls, you can't point it at an 5050f66f451Sopenharmony_ciarbitrary file out of the filesystem and have it run that. You could 5060f66f451Sopenharmony_ci"toybox nice /bin/ls" though.)</p> 5070f66f451Sopenharmony_ci 5080f66f451Sopenharmony_ci<hr /><h2><a name="standalone" />Q: How do I make individual/standalone toybox command binaries?</h2> 5090f66f451Sopenharmony_ci 5100f66f451Sopenharmony_ci<p>After running the configure step (generally "make defconfig") 5110f66f451Sopenharmony_ciyou can "make list" to see available command names you can use as build 5120f66f451Sopenharmony_citargets to build just that command 5130f66f451Sopenharmony_ci(ala "make sed"). Commands built this way do not contain a multiplexer and 5140f66f451Sopenharmony_cidon't care what the command filename is.</p> 5150f66f451Sopenharmony_ci 5160f66f451Sopenharmony_ci<p>The "make change" target (as in change for a $20) builds every command 5170f66f451Sopenharmony_cistandalone (in the "change" subdirectory). Note that this is collectively 5180f66f451Sopenharmony_ciabout 10 times as large as the multiplexer version, both in disk space and 5190f66f451Sopenharmony_ciruntime memory. (Even more when statically linked.)</p> 5200f66f451Sopenharmony_ci 5210f66f451Sopenharmony_ci<hr /><h2><a name="cross" />Q: How do I cross compile toybox?</h2> 5220f66f451Sopenharmony_ci 5230f66f451Sopenharmony_ci<p>A: You need a compiler "toolchain" capable of producing binaries that 5240f66f451Sopenharmony_cirun on your target. A toolchain is an 5250f66f451Sopenharmony_ciintegrated suite of compiler, assembler, and linker, plus the standard 5260f66f451Sopenharmony_ciheaders and 5270f66f451Sopenharmony_cilibraries necessary to build C programs. (And a few miscellaneous binaries like 5280f66f451Sopenharmony_cinm and objdump.)</p> 5290f66f451Sopenharmony_ci 5300f66f451Sopenharmony_ci<p>Toybox is tested against two compilers (llvm, gcc) and three C libraries 5310f66f451Sopenharmony_ci(bionic, musl, glibc) in the following combinations:</p> 5320f66f451Sopenharmony_ci 5330f66f451Sopenharmony_ci<a name="cross1" /> 5340f66f451Sopenharmony_ci<p><a href="#cross1">1) gcc+glibc = host toolchain</a></p> 5350f66f451Sopenharmony_ci 5360f66f451Sopenharmony_ci<p>Most Linux distros come with that as a host compiler, which is used by 5370f66f451Sopenharmony_cidefault when you build normally 5380f66f451Sopenharmony_ci(<b>make distclean defconfig toybox</b>, or <b>make menuconfig</b> followed 5390f66f451Sopenharmony_ciby <b>make</b>).</p> 5400f66f451Sopenharmony_ci 5410f66f451Sopenharmony_ci<p>You can use LDFLAGS=--static if you want static binaries, but static 5420f66f451Sopenharmony_ciglibc is hugely inefficient ("hello world" is 810k on x86-64) and throws a 5430f66f451Sopenharmony_cizillion linker warnings because one of its previous maintainers 5440f66f451Sopenharmony_ci<a href=https://www.akkadia.org/drepper/no_static_linking.html>was insane</a> 5450f66f451Sopenharmony_ci(which meant at the time he refused to fix 5460f66f451Sopenharmony_ci<a href=https://elinux.org/images/2/2d/ELC2010-gc-sections_Denys_Vlasenko.pdf>obvious bugs</a>), plus it uses dlopen() at runtime to implement basic things like 5470f66f451Sopenharmony_ci<a href=https://stackoverflow.com/questions/15165306/compile-a-static-binary-which-code-there-a-function-gethostbyname>DNS lookup</a> (which is almost impossible 5480f66f451Sopenharmony_cito support properly from a static binary because you wind up with two 5490f66f451Sopenharmony_ciinstances of malloc() managing two heaps which corrupt as soon as a malloc() 5500f66f451Sopenharmony_cifrom one is free()d into the other, although glibc added 5510f66f451Sopenharmony_ci<a href=https://stackoverflow.com/questions/14289488/use-dlsym-on-a-static-binary>improper support</a> which still requires the shared libraries to be 5520f66f451Sopenharmony_ciinstalled on the system alongside the static binary: 5530f66f451Sopenharmony_ci<a href=https://www.youtube.com/watch?v=Ih-3vK2qLls>in brief, avoid</a>). 5540f66f451Sopenharmony_ciThese days glibc is <a href=https://blog.aurel32.net/175>maintained 5550f66f451Sopenharmony_ciby a committee</a> instead of a single 5560f66f451Sopenharmony_cimaintainer, if that's an improvement. (As with Windows and 5570f66f451Sopenharmony_ciCobol, most people deal with it and get on with their lives.)</p> 5580f66f451Sopenharmony_ci 5590f66f451Sopenharmony_ci<a name="cross2" /> 5600f66f451Sopenharmony_ci<p><a href="#cross2">2) gcc+musl = musl-cross-make</a></p> 5610f66f451Sopenharmony_ci 5620f66f451Sopenharmony_ci<p>The cross compilers I test this with are built from the 5630f66f451Sopenharmony_ci<a href=http://musl.libc.org/>musl-libc</a> maintainer's 5640f66f451Sopenharmony_ci<a href=https://github.com/richfelker/musl-cross-make>musl-cross-make</a> 5650f66f451Sopenharmony_ciproject, built by running toybox's scripts/mcm-buildall.sh in that directory, 5660f66f451Sopenharmony_ciand then symlink the resulting "ccc" subdirectory into toybox where 5670f66f451Sopenharmony_ci"make root CROSS=" can find them, ala:</p> 5680f66f451Sopenharmony_ci 5690f66f451Sopenharmony_ci<blockquote><b><pre> 5700f66f451Sopenharmony_cicd ~ 5710f66f451Sopenharmony_cigit clone https://github.com/landley/toybox 5720f66f451Sopenharmony_cigit clone https://github.com/richfelker/musl-cross-make 5730f66f451Sopenharmony_cicd musl-cross-make 5740f66f451Sopenharmony_ci../toybox/scripts/mcm-buildall.sh # this takes a while 5750f66f451Sopenharmony_ciln -s $(realpath ccc) ../toybox/ccc 5760f66f451Sopenharmony_ci</pre></b></blockquote> 5770f66f451Sopenharmony_ci 5780f66f451Sopenharmony_ci<p>Instead of symlinking ccc, you can specify a CROSS_COMPILE= prefix 5790f66f451Sopenharmony_ciin the same format the Linux kernel build uses. You can either provide a 5800f66f451Sopenharmony_cifull path in the CROSS_COMPILE string, or add the appropriate bin directory 5810f66f451Sopenharmony_cito your $PATH. I.E:</p> 5820f66f451Sopenharmony_ci 5830f66f451Sopenharmony_ci<blockquote> 5840f66f451Sopenharmony_ci<b><p>make LDFLAGS=--static CROSS_COMPILE=~/musl-cross-make/ccc/m68k-linux-musl-cross/bin/m68k-linux-musl- distclean defconfig toybox</p></b> 5850f66f451Sopenharmony_ci</blockquote> 5860f66f451Sopenharmony_ci 5870f66f451Sopenharmony_ci<p>Is equivalent to:</p> 5880f66f451Sopenharmony_ci 5890f66f451Sopenharmony_ci<blockquote><b><p> 5900f66f451Sopenharmony_ciexport "PATH=~/musl-cross-make/ccc/m68k-linux-musl-cross/bin:$PATH"<br /> 5910f66f451Sopenharmony_ciLDFLAGS=--static make distclean defconfig toybox CROSS=m68k-linux-musl- 5920f66f451Sopenharmony_ci</p></b></blockquote> 5930f66f451Sopenharmony_ci 5940f66f451Sopenharmony_ci<p>Note: these examples use static linking becausae a dynamic musl binary 5950f66f451Sopenharmony_ciwon't run on your host unless you install musl's libc.so into the system 5960f66f451Sopenharmony_cilibraries (which is an accident waiting to happen adding a second C library 5970f66f451Sopenharmony_cito most glibc linux distribution) or play with $LD_LIBRARY_PATH. 5980f66f451Sopenharmony_ciIn theory you could "make root" a dynamic root filesystem with musl by copying 5990f66f451Sopenharmony_cithe shared libraries out of the toolchain, but I haven't bothered implementing 6000f66f451Sopenharmony_cithat in mkroot yet because a static linked musl hello world is 10k on x86 6010f66f451Sopenharmony_ci(5k if stripped).</p> 6020f66f451Sopenharmony_ci 6030f66f451Sopenharmony_ci<a name="cross3" /> 6040f66f451Sopenharmony_ci<p><a href="#cross3">3) llvm+bionic = Android NDK</a></p> 6050f66f451Sopenharmony_ci 6060f66f451Sopenharmony_ci<p>The <a href=https://developer.android.com/ndk/downloads>Android 6070f66f451Sopenharmony_ciNative Development Kit</a> provides an llvm toolchain with the bionic 6080f66f451Sopenharmony_cilibc used by Android. To turn it into something toybox can use, you 6090f66f451Sopenharmony_cijust have to add an appropriately prefixed "cc" symlink to the other 6100f66f451Sopenharmony_ciprefixed tools, ala:</p> 6110f66f451Sopenharmony_ci 6120f66f451Sopenharmony_ci<blockquote><b><pre> 6130f66f451Sopenharmony_ciunzip android-ndk-r21b-linux-x86_64.zip 6140f66f451Sopenharmony_cicd android-ndk-21b/toolchains/llvm/prebuilt/linux-x86_64/bin 6150f66f451Sopenharmony_ciln -s x86_64-linux-android29-clang x86_64-linux-android-cc 6160f66f451Sopenharmony_ciPATH="$PWD:$PATH" 6170f66f451Sopenharmony_cicd ~/toybox 6180f66f451Sopenharmony_cimake distclean 6190f66f451Sopenharmony_cimake LDFLAGS=--static CROSS_COMPILE=x86_64-linux-android- defconfig toybox 6200f66f451Sopenharmony_ci</pre></b></blockquote> 6210f66f451Sopenharmony_ci 6220f66f451Sopenharmony_ci<p>Again, you need to static link unless you want to install bionic on your 6230f66f451Sopenharmony_cihost. Binaries statically linked against bionic are almost as big as with 6240f66f451Sopenharmony_ciglibc, but at least it doesn't have the dlopen() issues. (You still can't 6250f66f451Sopenharmony_cisanely use dlopen() from a static binary, but bionic doesn't use dlopen() 6260f66f451Sopenharmony_ciinternally to implement basic features.)</p> 6270f66f451Sopenharmony_ci 6280f66f451Sopenharmony_ci<p>Note: although the resulting toybox will run in a standard 6290f66f451Sopenharmony_ciLinux system, even "hello world" 6300f66f451Sopenharmony_cistatically linked against bionic segfaults before calling main() 6310f66f451Sopenharmony_ciwhen /dev/null isn't present. This presents mkroot with a chicken and 6320f66f451Sopenharmony_ciegg problem for both chroot and qemu cases, because mkroot's init script 6330f66f451Sopenharmony_cihas to mount devtmpfs on /dev to provide /dev/null before the shell binary 6340f66f451Sopenharmony_cican run mkroot's init script. 6350f66f451Sopenharmony_ciSince mkroot runs as a normal user, we can't "mknod dev/null" at build 6360f66f451Sopenharmony_citime to create a "null" device in the filesystem we're packaging up so 6370f66f451Sopenharmony_ciinitramfs doesn't start with an empty /dev, and the 6380f66f451Sopenharmony_ci<a href=https://lkml.org/lkml/2016/6/22/686>kernel</a> 6390f66f451Sopenharmony_ci<a href=https://lkml.org/lkml/2017/5/14/180>developers</a> 6400f66f451Sopenharmony_ci<a href=https://lkml.org/lkml/2017/9/13/651>repeatedly</a> 6410f66f451Sopenharmony_ci<a href=https://lkml.org/lkml/2020/5/14/1584>rejected</a> a patch to 6420f66f451Sopenharmony_cimake the Linux kernel honor DEVTMPFS_MOUNT in initramfs. Teaching toybox 6430f66f451Sopenharmony_cicpio to accept synthetic filesystem metadata, 6440f66f451Sopenharmony_cipresumably in <a href=https://www.kernel.org/doc/Documentation/filesystems/ramfs-rootfs-initramfs.txt>get_init_cpio</a> format, remains a todo item.</p> 6450f66f451Sopenharmony_ci 6460f66f451Sopenharmony_ci<hr /><h2><a name="system" />Q: What part of Linux/Android does toybox provide?</h2> 6470f66f451Sopenharmony_ci 6480f66f451Sopenharmony_ci<p>A: 6490f66f451Sopenharmony_ciToybox is one of three packages (linux, libc, command line) which together provide a bootable unix-style command line operating system. 6500f66f451Sopenharmony_ciToybox provides the "command line" part, with a 6510f66f451Sopenharmony_ci<a href=https://en.wikipedia.org/wiki/Bash_(Unix_shell)>bash</a> compatible 6520f66f451Sopenharmony_ci<a href=https://en.wikipedia.org/wiki/Unix_shell>command line interpreter</a> 6530f66f451Sopenharmony_ciand over two hundred <a href=https://landley.net/toybox/help.html>commands</a> 6540f66f451Sopenharmony_cito call from it, as documented in 6550f66f451Sopenharmony_ci<a href=https://pubs.opengroup.org/onlinepubs/9699919799.2008edition/>posix</a>, 6560f66f451Sopenharmony_cithe <a href=https://refspecs.linuxfoundation.org/LSB_4.1.0/LSB-Core-generic/LSB-Core-generic/cmdbehav.html>Linux Standard Base</a>, and the 6570f66f451Sopenharmony_ci<a href=https://man7.org/linux/man-pages/dir_section_1.html>Linux Manual 6580f66f451Sopenharmony_ciPages</a>.</p> 6590f66f451Sopenharmony_ci 6600f66f451Sopenharmony_ci<p>Toybox is not by itself a complete operating system, it's a set of standard command line utilities that run in an operating system. 6610f66f451Sopenharmony_ciBooting a simple system to a shell prompt requires a kernel to drive the hardware (such as Linux, or BSD with a Linux emulation layer), programs for the system to run (such as toybox's commands), and a C library ("libc") to connect them together.</p> 6620f66f451Sopenharmony_ci 6630f66f451Sopenharmony_ci<p>Toybox has a policy of requiring no external dependencies other than the 6640f66f451Sopenharmony_cikernel and C library (at least for defconfig builds). You can optionally enable support for 6650f66f451Sopenharmony_ciadditional libraries in menuconfig (such as openssl, zlib, or selinux), 6660f66f451Sopenharmony_cibut toybox either provides its own built-in versions of such functionality 6670f66f451Sopenharmony_ci(which the libraries provide larger, more complex, often assembly optimized 6680f66f451Sopenharmony_cialternatives to), or allows things like selinux support to cleanly drop 6690f66f451Sopenharmony_ciout.</p> 6700f66f451Sopenharmony_ci 6710f66f451Sopenharmony_ci<p>Static linking (with the --static option) copies library contents 6720f66f451Sopenharmony_ciinto the resulting binary, creating larger but more portable programs which 6730f66f451Sopenharmony_cican run even if they're the only file in the filesystem. Otherwise, 6740f66f451Sopenharmony_cithe "dynamically" linked programs require each shared library file to be 6750f66f451Sopenharmony_cipresent on the target system, either copied out of the toolchain or built 6760f66f451Sopenharmony_ciagain from source (with potential version skew if they don't match the toolchain 6770f66f451Sopenharmony_civersions exactly), plus a dynamic linker executable installed at a specific 6780f66f451Sopenharmony_ciabsolute path. See the 6790f66f451Sopenharmony_ci<a href=https://www.man7.org/linux/man-pages/man1/ldd.1.html>ldd</a>, 6800f66f451Sopenharmony_ci<a href=https://www.man7.org/linux/man-pages/man8/ld.so.8.html>ld.so</a>, 6810f66f451Sopenharmony_ciand <a href=https://www.man7.org/linux/man-pages/man7/libc.7.html>libc</a> 6820f66f451Sopenharmony_ciman pages for details.</p> 6830f66f451Sopenharmony_ci 6840f66f451Sopenharmony_ci<p>Most embedded systems will add another package to the kernel/libc/cmdline 6850f66f451Sopenharmony_ciabove containing the dedicated "application" that the embedded system exists to 6860f66f451Sopenharmony_cirun, plus any other packages that application depends on. 6870f66f451Sopenharmony_ciBuild systems add a native version of the toolchain packages so 6880f66f451Sopenharmony_cithey can compile additional software on the resulting system. Desktop systems 6890f66f451Sopenharmony_ciadd a GUI and additional application packages like web browsers 6900f66f451Sopenharmony_ciand video players. A linux distro like Debian adds hundreds of packages. 6910f66f451Sopenharmony_ciAndroid adds around a thousand.</p> 6920f66f451Sopenharmony_ci 6930f66f451Sopenharmony_ci<p>But all of these systems conceptually sit on a common three-package 6940f66f451Sopenharmony_ci"kernel/libc/cmdline" base (often inefficiently implemented and broken up 6950f66f451Sopenharmony_ciinto more packages), and toybox aims to provide a simple, reproducible, 6960f66f451Sopenharmony_ciauditable version of the cmdline portion of that base.</p> 6970f66f451Sopenharmony_ci 6980f66f451Sopenharmony_ci<hr /><h2><a name="mkroot" />Q: How do you build a working Linux system with toybox?</h2> 6990f66f451Sopenharmony_ci 7000f66f451Sopenharmony_ci<p>A: Toybox has a built-in <a href=https://github.com/landley/toybox/blob/master/scripts/mkroot.sh>system builder</a>, with the Makefile target "<b>make 7010f66f451Sopenharmony_ciroot</b>". To enter the resulting root filesystem, "<b>sudo chroot 7020f66f451Sopenharmony_ciroot/host/fs /init</b>". Type "exit" to get back out.</p> 7030f66f451Sopenharmony_ci 7040f66f451Sopenharmony_ci<p>You can cross compile simple three package (toybox+libc+linux) 7050f66f451Sopenharmony_cisystems configured to boot to a shell prompt under the emulator 7060f66f451Sopenharmony_ci<a href=https://qemu.org>qemu</a> 7070f66f451Sopenharmony_ciby specifying a target type with CROSS= 7080f66f451Sopenharmony_ci(or by setting CROSS_COMPILE= to a <a href=#cross>cross compiler</a> prefix with optional absolute 7090f66f451Sopenharmony_cipath), and pointing the build at a Linux kernel source directory, ala:</p> 7100f66f451Sopenharmony_ci 7110f66f451Sopenharmony_ci<blockquote><p><b>make root CROSS=sh4 LINUX=~/linux</b></p></blockquote> 7120f66f451Sopenharmony_ci 7130f66f451Sopenharmony_ci<p>Then you can <b>cd root/sh4; ./qemu-sh4.sh</b> to launch the emulator. 7140f66f451Sopenharmony_ci(You'll need the appropriate qemu-system-* emulator binary installed.) 7150f66f451Sopenharmony_ciType "exit" when done and it should shut down the emulator on the way out, 7160f66f451Sopenharmony_cisimilar to exiting the chroot version. (Except this is more like you ssh'd 7170f66f451Sopenharmony_cito a remote machine: the emulator created its own CPU with its own memory 7180f66f451Sopenharmony_ciand I/O devices, and booted a kernel in it.)</p> 7190f66f451Sopenharmony_ci 7200f66f451Sopenharmony_ci<p>The build finds the <a href=#system>three packages</a> needed to produce 7210f66f451Sopenharmony_cithis system because 1) you're in a toybox source directory, 2) your cross 7220f66f451Sopenharmony_cicompiler has a libc built into it, 3) you tell it where to find a Linux kernel 7230f66f451Sopenharmony_cisource directory with LINUX= on the command line. If you don't say LINUX=, 7240f66f451Sopenharmony_ciit skips that part of the build and just produces a root filesystem directory 7250f66f451Sopenharmony_ciala the first example in this FAQ answer.</p> 7260f66f451Sopenharmony_ci 7270f66f451Sopenharmony_ci<p>The CROSS= shortcut expects a "ccc" symlink in the toybox source directory 7280f66f451Sopenharmony_cipointing at a directory full of cross compilers. The ones I test this with are built from the musl-libc 7290f66f451Sopenharmony_cimaintainer's 7300f66f451Sopenharmony_ci<a href=https://github.com/richfelker/musl-cross-make>musl-cross-make</a> 7310f66f451Sopenharmony_ciproject, built by running toybox's scripts/mcm-buildall.sh in that directory, 7320f66f451Sopenharmony_ciand then symlink the resulting "ccc" subdirectory into toybox where CROSS= 7330f66f451Sopenharmony_cican find them:</p> 7340f66f451Sopenharmony_ci 7350f66f451Sopenharmony_ci<blockquote><b><pre> 7360f66f451Sopenharmony_cicd ~ 7370f66f451Sopenharmony_cigit clone https://github.com/landley/toybox 7380f66f451Sopenharmony_cigit clone https://github.com/richfelker/musl-cross-make 7390f66f451Sopenharmony_cicd musl-cross-make 7400f66f451Sopenharmony_ci../toybox/scripts/mcm-buildall.sh # this takes a while 7410f66f451Sopenharmony_ciln -s $(realpath ccc) ../toybox/ccc 7420f66f451Sopenharmony_ci</pre></b></blockquote> 7430f66f451Sopenharmony_ci 7440f66f451Sopenharmony_ci<p>If you don't want to do that, you can download <a href=http://mkroot.musl.cc/latest/>prebuilt binary versions</a> from Zach van Rijn's site and 7450f66f451Sopenharmony_cijust extract them into a "ccc" subdirectory under the toybox source.</p> 7460f66f451Sopenharmony_ci 7470f66f451Sopenharmony_ci<p>Once you've installed the cross compilers, "<b>make root CROSS=help</b>" 7480f66f451Sopenharmony_cishould list all the available cross compilers it recognizes under ccc, 7490f66f451Sopenharmony_cisomething like:</p> 7500f66f451Sopenharmony_ci 7510f66f451Sopenharmony_ci<blockquote><b><p> 7520f66f451Sopenharmony_ciaarch64 armv4l armv5l armv7l armv7m armv7r i486 i686 m68k microblaze mips mips64 mipsel powerpc powerpc64 powerpc64le s390x sh2eb sh4 x32 x86_64 7530f66f451Sopenharmony_ci</p></b></blockquote> 7540f66f451Sopenharmony_ci 7550f66f451Sopenharmony_ci<p>(A long time ago I 7560f66f451Sopenharmony_ci<a href=http://landley.net/aboriginal/architectures.html>tried to explain</a> 7570f66f451Sopenharmony_ciwhat some of these architectures were.)</p> 7580f66f451Sopenharmony_ci 7590f66f451Sopenharmony_ci<p>You can build all the targets at once, and can add additonal packages 7600f66f451Sopenharmony_cito the build, by calling the script directly and listing packages on 7610f66f451Sopenharmony_cithe command line:</p> 7620f66f451Sopenharmony_ci 7630f66f451Sopenharmony_ci<blockquote> 7640f66f451Sopenharmony_ci<p><b>scripts/mkroot.sh CROSS=all LINUX=~/linux dropbear</b></p> 7650f66f451Sopenharmony_ci</blockquote> 7660f66f451Sopenharmony_ci 7670f66f451Sopenharmony_ci<p>An example package build script (building the dropbear ssh server, adding a 7680f66f451Sopenharmony_ciport forward from 127.0.0.1:2222 to the qemu command line, and providing a 7690f66f451Sopenharmony_cissh2dropbear.sh convenience script to the output directory) is provided 7700f66f451Sopenharmony_ciin the scripts/root directory. If you add your own scripts elsewhere, just 7710f66f451Sopenharmony_cigive a path to them on the command line. (No, I'm not merging more package build 7720f66f451Sopenharmony_ciscripts, I <a href=https://speakerdeck.com/landley/developing-for-non-x86-targets-using-qemu?slide=78>learned that lesson</a> long ago. But if you 7730f66f451Sopenharmony_ciwant to write your own, feel free.)</p> 7740f66f451Sopenharmony_ci 7750f66f451Sopenharmony_ci<p>(Note: currently mkroot.sh cheats. If you don't have a .config it'll 7760f66f451Sopenharmony_cimake defconfig and add CONFIG_SH and CONFIG_ROUTE to it, because the new 7770f66f451Sopenharmony_ciroot filesystem kinda needs those commands to function properly. If you already 7780f66f451Sopenharmony_cihave a .config that 7790f66f451Sopenharmony_ci_doesn't_ have CONFIG_SH in it, you won't get a shell prompt or be able to run 7800f66f451Sopenharmony_cithe init script without a shell. This is currently a problem because sh 7810f66f451Sopenharmony_ciand route are still in pending and thus not in defconfig, so "make root" 7820f66f451Sopenharmony_cicheats and adds them. I'm working on it. tl;dr if make root doesn't work 7830f66f451Sopenharmony_ci"rm .config" and run it again, and all this should be fixed up in future when 7840f66f451Sopenharmony_cithose two commands are promoted out of pending so "make defconfig" would have 7850f66f451Sopenharmony_ciwhat you need anyway. It's designed to let yout tweak your config, which is 7860f66f451Sopenharmony_ciwhy it uses the .config that's there when there is one, but the default is 7870f66f451Sopenharmony_cicurrently wrong because it's not quite finished yet. All this should be 7880f66f451Sopenharmony_cicleaned up in a future release, before 1.0.)</p> 7890f66f451Sopenharmony_ci 7900f66f451Sopenharmony_ci 7910f66f451Sopenharmony_ci 7920f66f451Sopenharmony_ci<!--#include file="footer.html" --> 793