I am typing this on an editor running on Debian on my Samsung Galaxy S Relay 4G, aka apexqtmo. Sorry if I got you the wrong image. I am not using its touchscreen or keyboard for input or its screen for output. And the editor is not graphical. I am using a console editor over an SSH session.
I will try to summarize the steps needed to get there, what else I got, the challenges that I had to overcome, and some of the challenges ahead.
My first challenge was to get some output so that I could debug any problems. If I had a terminal, even better. So, this device has a keyboard, which makes things easier for input. I once had it running with a small initrd with GNU bash and got output on the framebuffer, though a little garbled. I realize there are some problems with the framebuffer driver after I have done some tests, which would explain that. For some reason, that didn't work again for me for some time.
So, in the effort to get some debug output, I have spent a lot of time trying to get a Linux binary that would manage to output to a framebuffer console, but without no luck. Then, I realized that besides ADB, I could just configure the USB gadget to act like a serial ACM device, and have a serial console. After that, I have made a lot of progress.
So, I had a small init which would do the configuration, then mount an SD card partition as the root filesystem and have bash for me. Some of the first challenges here were the fact that this old kernel didn't support metadata checksum on ext4, so I had to tune or recreate the filesystem.
After a success with bash, I went for init. As I had a debootstrapped Debian Jessie, that would be systemd, which failed for no known reason yet, but that was expected. Though this was a 3.4 Linux, there are many missing features in the binary which would explain it. So, I went for System V, and after setting it up to have a getty at ttyGS0, I had a login console.
So, another problem is that udev initscript requires devtmpfs, which this Linux binary didn't have enabled. I decided to ignore that and skip udev. I had created some static nodes at /dev/, which were good enough for me.
Then, for the network, I noticed this kernel didn't have wireless as well, so I booted with a different one, but it looks like it requires some extra initialization, maybe some userspace loading firmware, so I skipped that as well. Now, as for a network connection, I then thought I could use the same USB strategy and have it as a multiple interface gadget, with both serial ACM and network functions. That worked wonders.
Now, I could SSH into the smartphone and start running apt, and install packages. You know what's the next step? Get that framebuffer working and have X.org running. More work getting back to a kernel where framebuffer worked. Lots of headaches for that. Though there was a framebuffer driver, it wouldn't work for the kernel that is used for CyanogenMod. It seems it requires OpenGL to work. Or some fiddling with sysfs that I couldn't mimick in order for the display to turn on. So, back to a different kernel, where things seemed to work somehow.
Now, as udev was not running, X.org would give me something at the framebuffer, but no input would work. After trying to configure it manually, change some settings here and there, I found out that only the init scripts checked for the presence of devtmpfs. So, I removed some part of that, and udev was on. But that was after an upgrade to sid, so that might have been necessary, though I can't tell for sure.
For some reason, some X clients didn't like when I used nodm as a display manager, so I went for xinit. And the X server crashed when I used certain clients. I have a hunch this is caused by a framebuffer driver bug in the kernel, which would be a very sad fact if true. So, I got xterm and matchbox-keyboard running. The touchscreen and keyboard worked fine, so now I had something to show off. Though I was excited already when I had "Hello, World!" at my USB serial console.
So, now we realize what a problem we have! These binary kernels, even the ones built by a community like CyanogenMod, require a userspace in order to initialize drivers, including OpenGL and wireless. They don't have basic features turned on, like devtmpfs. It's a daunting task to run a different userspace, that might require these features or not have any code to do those required initializations.
It's still doable to some point, but maybe not sustainable for long. We would either have to work on an alternative userspace, one that does not require or can work without some kernel features, and that can do some of the initialization or workaround some of the bugs we find in the drivers, like the framebuffer one.
The best alternative is to get those devices supported upstream. And, for that, the first step is to get some source code that builds and runs on those things. I didn't manage to get there yet. After some failures with what is supposed to be used for CyanogenMod, I tried getting source code directly from Samsung. As I couldn't find my device in their opensource site, I asked them for the source code. Four days later, they sent me a message that the source was now available in their site. That was a nice surprise. I didn't expect them to answer, and that was quick enough for me to try it before I was about to present this topic at Latinoware. It turns out I couldn't boot that yet either, and it is a 3.0 Linux, so old enough not to be supported by the latest Debian.
Next I am going to write how to try the same thing for your own Samsung gadget at your own risk, no warranties.