Read binary stdout data like screencap data from adb shell?

Question

Is it possible to read binary stdout from an adb shell command? For example, all examples of how to use screencap include two steps:

adb shell screencap -p /sdcard/foo.png
adb pull /sdcard/foo.png

However, the service supports writing to stdout. You can for instance, do the following:

adb shell "screencap -p > /sdcard/foo2.png"
adb pull /sdcard/foo2.png

And this works equally well. But, what about reading the output across ADB? What I want to do is the following:

adb shell screencap -p > foo3.png

And avoid the intermediate write to the SD card. This generates something that looks like a PNG file (running strings foo3.png generates something with an IHDR, IEND, etc.) and is approximately the same size, but the file is corrupted as far as image readers are concerned.

I have also attempted to do this using ddmlib in java and the results are the same. I would be happy to use any library necessary. My goal is to reduce total time to get the capture. On my device, using the two-command solution, it takes about 3 seconds to get the image. Using ddmlib and capturing stdout takes less than 900ms, but it doesn't work!

Is it possible to do this?

EDIT: Here is the hexdump of two files. The first one, screen.png came from stdout and is corrupted. The second one, xscreen is from the two-command solution and works. The images should be visually identical.

$ hexdump -C screen.png | head
00000000  89 50 4e 47 0d 0d 0a 1a  0d 0a 00 00 00 0d 49 48  |.PNG..........IH|
00000010  44 52 00 00 02 d0 00 00  05 00 08 06 00 00 00 6e  |DR.............n|
00000020  ce 65 3d 00 00 00 04 73  42 49 54 08 08 08 08 7c  |.e=....sBIT....||
00000030  08 64 88 00 00 20 00 49  44 41 54 78 9c ec bd 79  |.d... .IDATx...y|
00000040  9c 1d 55 9d f7 ff 3e 55  75 f7 de b7 74 77 d2 d9  |..U...>Uu...tw..|
00000050  bb b3 27 10 48 42 16 c0  20 01 86 5d 14 04 11 dc  |..'.HB.. ..]....|
00000060  78 44 9d c7 d1 d1 11 78  70 7e 23 33 8e 1b 38 33  |xD.....xp~#3..83|
00000070  ea 2c 8c 8e 0d 0a 08 a8  23 2a 0e 10 82 ac c1 40  |.,......#*.....@|
00000080  12 02 81 24 64 ef ec 5b  ef fb 5d 6b 3b bf 3f ea  |...$d..[..]k;.?.|
00000090  de db dd 49 27 e9 ee 74  77 3a e3 79 bf 5e 37 e7  |...I'..tw:.y.^7.|

$ hexdump -C xscreen.png | head
00000000  89 50 4e 47 0d 0a 1a 0a  00 00 00 0d 49 48 44 52  |.PNG........IHDR|
00000010  00 00 02 d0 00 00 05 00  08 06 00 00 00 6e ce 65  |.............n.e|
00000020  3d 00 00 00 04 73 42 49  54 08 08 08 08 7c 08 64  |=....sBIT....|.d|
00000030  88 00 00 20 00 49 44 41  54 78 9c ec 9d 77 98 1c  |... .IDATx...w..|
00000040  c5 99 ff 3f d5 dd 93 37  27 69 57 5a e5 55 4e 08  |...?...7'iWZ.UN.|
00000050  24 a1 00 58 18 04 26 08  8c 01 83 31 38 c0 19 9f  |$..X..&....18...|
00000060  ef 7c c6 3e 1f 70 f8 7e  67 ee 71 e2 b0 ef ce f6  |.|.>.p.~g.q.....|
00000070  f9 ec 73 04 1b 1c 31 60  23 84 30 22 88 a0 40 10  |..s...1`#.0"..@.|
00000080  08 65 69 95 d3 4a 9b c3  c4 4e f5 fb a3 67 66 77  |.ei..J...N...gfw|
00000090  a5 95 b4 bb da a4 73 7d  9e 67 55 f3 ed 50 5d dd  |......s}.gU..P].|

Just at quick glance it seems like a couple of extra 0x0d (13) bytes get added. Carriage return?? Does that ring any bells? Is it mixing in some blank lines?

Answer 1

Unlike adb shell the adb exec-out command doesn't use pty which mangles the binary output. So you can do

adb exec-out screencap -p > test.png

https://android.googlesource.com/platform/system/core/+/5d9d434efadf1c535c7fea634d5306e18c68ef1f

Note that if you are using this technique for a command that produces output on STDERR, you should redirect it to /dev/null, otherwise adb will include STDERR in its STDOUT corrupting your output. For example, if you are trying to backup and compress a directory:

adb exec-out "tar -zcf - /system 2>/dev/null" > system.tar.gz

Answer 2

Sorry to be posting an answer to an old question, but I just came across this problem myself and wanted to do it only through the shell. This worked well for me:

adb shell screencap -p | sed 's/^M$//' > screenshot.png

That ^M is a char I got by pressing ctrl+v -> ctrl+m, just noticed it doesn't work when copy-pasting.

adb shell screencap -p | sed 's/\r$//' > screenshot.png

did the trick for me as well.

Answer 3

As noted, "adb shell" is performing a linefeed (0x0a) to carriage-return + linefeed (0x0d 0x0a) conversion. This is being performed by the pseudo-tty line discipline. As there is no "stty" command available to the shell, there is no easy way to mess with the terminal settings.

It's possible to do what you want with ddmlib. You'd need to write code that executed commands on the device, captured the output, and sent it over the wire. This is more or less what DDMS does for certain features. This may be more trouble than its worth.

The repair() solution -- converting all CRLF to LF -- feels shaky but is actually reliable since the "corrupting" LF-to-CRLF conversion is deterministic. I used to do the same thing to repair inadvertent ASCII-mode FTP transfers.

It's worth noting that the PNG file format is explicitly designed to catch exactly this (and related) problems. The magic number begins with 0x89 to catch anything that strips high bits, followed by "PNG" so you can easily tell what's in the file, followed by CR LF to catch various ASCII line converters, then 0x1a to trap old MS-DOS programs that used Ctrl-Z as a special end-of-file marker, and then a lone LF. By looking at the first few bytes of the file you can tell exactly what was done to it.

...which means that your repair() function can accept both "corrupted" and "pure" input, and reliably determine if it needs to do anything.

Edit: one additional note: it's possible for the device-side binary to configure the tty to avoid the conversion, using cfmakeraw(). See the prepareRawOutput() function in the screenrecord command in Android 5.0, which can send raw video from the live screen capture across the ADB shell connection.

Answer 4

The best solution is to use adb exec-out command like @AjeetKhadke suggested.

Let me illustrate the difference between adb shell and adb exec-out output:

~$ adb shell "echo -n '\x0a'" | xxd -g1
00000000: 0d 0a

~$ adb exec-out "echo -n '\x0a'" | xxd -g1
00000000: 0a

It works in Windows (I am using hexdump from GNUWin32 Hextools for the demo) as well:

C:\>adb shell "echo -n '\x0a'" | hexdump
00000000: 0D 0A

C:\>adb exec-out "echo -n '\x0a'" | hexdump
00000000: 0A

The downside is that in order to be able to benefit from using the adb exec-out command both the device and host PC have to support adb shell V2 protocol.

It is rather trivial to take care of the PC side - just update the platform-tools package (which contains the adb binary) to the latest version. The version of adbd daemon on the device is linked to the version of Android. The adb shell V2 protocol has been introduced in Android 5.0 together with complete adb overhaul (going from c to C++ code). But there were some regressions (aka bugs) so adb exec-out usefulness in Android 5.x was still limited. And finally there is no support for Android 4.x and older devices. Fortunately the share of those older devices still being used for development is dropping fast.

Answer 5

After digging deeper into the hex dumps it became clear that every time the character 0x0A was emitted, the shell would emit 0x0D 0x0A. I repaired the stream with the following code and now the binary data is correct. Now, of course, the question is why is adb shell doing this? But in any event, this fixes the problem.

static byte[] repair(byte[] encoded) {
    ByteArrayOutputStream baos = new ByteArrayOutputStream();
    for (int i=0; i<encoded.length; i++) {
        if (encoded.length > i+1 && encoded[i] == 0x0d && encoded[i+1] == 0x0a) {
            baos.write(0x0a);
            i++;
        } else {
            baos.write(encoded[i]);
        }
    }
    try {
        baos.close();
    } catch (IOException ioe) {

    }

    return baos.toByteArray();      
}

EDIT: It dawned on me why it is doing this. It is converting LF to CR/LF like old-school DOS. I wonder if there is a setting somewhere to turn that off?

Answer 6

Yes, on Unix/Linux/Mac OS X, you can receive binary output of adb shell by prepend "stty -onlcr;" to your command ( NO~~ need to be a rooted android).

1.Download "stty" executable file.
http://www.busybox.net/downloads/binaries/latest/
For old android, use busybox-armv5l, Others use busybox-armv7l.
rename file to "stty"

2.Uploda file "stty" to android and set proper permission.

adb push somelocaldir/stty /data/local/tmp/   
adb shell chmod 777 /data/local/tmp/stty 

3.Prepend "stty -onlcr;" to your command like this;

adb shell /data/local/tmp/stty -onlcr\; screencap -p  > somelocaldir/output.png
or:
adb shell "/data/local/tmp/stty -onlcr; screencap -p"  > somelocaldir/output.png
or (Only for Windows):
adb shell /data/local/tmp/stty -onlcr; screencap -p  > somelocaldir/output.png

Done!

But for Windows OS, by default, LF from android will be converted to CR CR LF.
Even you did above step, you still get CR LF.
This "seems" because local adb.exe use fwrite which cause CR be prepended.
I have no way about this except convert CR LF to LF manually on Windows OS.

Answer 7

Here is solution that works everywhere (Linux and Windows included).

You will need netcat utility, often named nc.
If both nc and busybox nc fail on your device, you need fresh busybox. You can either use busybox installer from Play Market (root required), or use solution by osexp2003 (download busybox from official site, put it into /data/local/tmp/ on device and add execute permission).

The idea is to use netcat as a primitive HTTP server.
Well, not even a proper server in fact. It will just send its input as response to any TCP connection (be it HTTP request from browser, telnet connection or just netcat) and terminate.

Run command you want to get output from like this:

adb shell 'screencap -p | busybox nc -p 8080 -l >/dev/null'

In the above example, screencap -p takes a screenshot (PNG image) and pipes it to netcat.
-l tells netcat to act as a server (listen for connection), and -p 8080 tells it to use TCP port 8080. Omiting >/dev/null will simply print e.g. incoming HTTP GET request to your terminal.
The above example will wait for someone to connect, send screenshot and only then terminate.
Of course you can run it without adb shell, e.g. from terminal emulator on your device.

After running your command as above, you can download its output from your phone, by opening http://ip.of.your.phone:8080 in browser or by any other means, for example using netcat:

busybox nc ip.of.your.phone:8080 >screenshot.png

If you want to use USB cable for download, you need to forward connection using ADB like this:

adb forward tcp:7080 tcp:8080

After that you can use localhost:7080 instead of ip.of.your.phone:8080.
You can remove this forwarding with following command:

adb forward --remove tcp:7080

Answer 8

Another way:

adb shell "busybox stty raw; screencap -p "> foo3.png 

BUT, as @osexp2003 said, that does not work for Windows OS.

Answer 9

It is old question but may be this solution will be usefull for somebody.

For simply downloading any files from adb you can use:

adb run-as your.package.name base64 -w 0 /path/to/your/file.db

and then just read and decode base64 string in linux (as example):

cat saved.base64.str|base64 -d

Of course you can compress source data before encode it as base64.

Answer 10

You can also use the standard dos2unix command if its available.

(apt-get install dos2unix if you're on Debian/Ubuntu. There are probably builds for Windows, OS X, etc. out there somewhere if you google).

dos2unix converts CRLF to LF the same way as Eric Lange's repair() function.

adb shell screencap -p | dos2unix -f > screenshot.png

or, fix a corrupted file (in-place) :

dos2unix -f screenshot.png

You need the -f to force it to process binary files.

Answer 11

It is also possible to use base64 for this, so just encode it using:

base64 foo3.png>foo3.png.base64

and then on windows using some base64 utility or maybe notepad++ to decrypt the file.

Or in linux / cygwin:

base64 -d foo3.png.base64>foo3.png

Answer 12

try this guys:

adb shell screencap -p | perl -pe 's/\x0D\x0A/\x0A/g' > screen.png

Answer 13

I put the method to use python get image bytes using adb here, maybe this will be helpful to someone who encountered this problem. The code is as following:

 pipe = subprocess.Popen("adb shell screencap -p",
                      stdin=subprocess.PIPE,
                      stdout=subprocess.PIPE, shell=True)
 image_bytes = pipe.stdout.read().replace(b'\r\n', b'\n')
 gray_image = cv2.imdecode(np.fromstring(image_bytes, np.uint8), cv2.IMREAD_GRAYSCALE)

Answer 14

The way to go in general is to use adb exec-out, as Ajeet47 and Alex P. pointed out.

However, things get messy if there is anything in-between which creates a pseudo-tty.

I wondered why my tar files got corrupted even though I was using adb exec-out. The issue was using the su binary to spawn tar processes in my case. su seems to spawn a pseudo-tty and you can't turn that off, giving you the worst of both worlds.

Example:

% adb exec-out 'su -c '"'"'printf "\n"'"'" | xxd
00000000: 0d0a                                     ..

In such scenarios, you still need to tell the shell to not mangle line feeds with additional carriage return characters through means like stty raw:

% adb exec-out 'su -c '"'"'stty raw; printf "\n"'"'" | xxd                                                                  
00000000: 0a                                       .

However, as others pointed out, this might be fragile, especially for binary data. I haven't seen any issues when receiving tar output over adb when pairing exec-out with stty raw yet, but there certainly could be issues.

Hence, if you really want bit-for-bit equality, check if the generated data matches what you expect to catch surprising situations. This will not fix them, but at least alert you to them.

One way to do this is to checksum the original data while writing it out and checksum the received data on the host. A rather convoluted, but necessary, way to do this with POSIX-compatible systems and shells goes like that:

% adb exec-out 'su -c '"'"'stty raw && printf "\n" 2>/data/local/tmp/printf.stderr | { tee /dev/fd/3 | sha512sum -b - > /data/local/tmp/printf.sha512sum; } 3>&1'"'" | sha512sum -b - 
be688838ca8686e5c90689bf2ab585cef1137c999b48c70b92f67a5c34dc15697b5d11c982ed6d71be1e1e7f7b4e0733884aa97c3f7a339a8ed03577cf74be09 *-
% adb exec-out 'cat /data/local/tmp/printf.sha512sum'
be688838ca8686e5c90689bf2ab585cef1137c999b48c70b92f67a5c34dc15697b5d11c982ed6d71be1e1e7f7b4e0733884aa97c3f7a339a8ed03577cf74be09

Do not be tempted to use bash's command line substitution feature (such as cmd | tee >(sha512sum -b - > ...)) since that will sadly drop a trailing newline and hence modify the original data.

Answer 15

nc was the only way it worked for me. Used:

adb forward tcp:7080 tcp:8080 &&\    
adb shell 'tar -zcvf - /data/media | nc -p 8080 -l 1>/dev/null' &\    
sleep 1;\    
nc localhost 7080 > media.tar.gz &&\    
adb forward --remove tcp:7080

as root to create a hopefully proper backup for /data/media

Answer 16

I want to add another solution for those apps, that prevent adb from taking screenshots.

You can use scrcpy for that. It works on Linux and Windows! Just run:

scrcpy

A windows with your device screen will open. You can now use your local screenshot tool (e.g. snipping tool on windows and screenshot on linux) to take a screenshot of the current screen!

Answer 17

You can use this command and saves in the active directory

adb exec-out screencap -p > screen-shot.png

Answer 18

This command worked for me on Windows OS:

adb exec-out screencap -p > test.png && dos2unix.exe -f test.png

But you want to use this: https://sourceforge.net/projects/dos2unix/

Answer 19

This is the best way using Shell in OS

adb shell screencap -p | perl -pe 's/\x0D\x0A/\x0A/g' > screen.png