get all keys set in memcached

Question

How can I get all the keys set in my memcached instance(s)?

I tried googling, but didn't find much except that PHP supports a getAllKeys method, which means it is actually possible to do this somehow. How can I get the same within a telnet session?

I have tried out all the retrieval related options mentioned in memcached cheat sheet and Memcached telnet command summary, but none of them work and I am at a loss to find the correct way to do this.

Note: I am currently doing this in development, so it can be assumed that there will be no issues due to new keys being set or other such race conditions happening, and the number of keys will also be limited.

Answer 1

Found a way, thanks to the link here (with the original google group discussion here)

First, Telnet to your server:

telnet 127.0.0.1 11211

Next, list the items to get the slab ids:

stats items
STAT items:3:number 1
STAT items:3:age 498
STAT items:22:number 1
STAT items:22:age 498
END

The first number after ‘items’ is the slab id. Request a cache dump for each slab id, with a limit for the max number of keys to dump:

stats cachedump 3 100
ITEM views.decorators.cache.cache_header..cc7d9 [6 b; 1256056128 s]
END

stats cachedump 22 100
ITEM views.decorators.cache.cache_page..8427e [7736 b; 1256056128 s]
END

Answer 2

memdump

There is a memcdump (sometimes memdump) command for that (part of libmemcached-tools), e.g.:

memcdump --servers=localhost

which will return all the keys.

---

memcached-tool

In the recent version of memcached there is also memcached-tool command, e.g.

memcached-tool localhost:11211 dump | less

which dumps all keys and values.

See also:

• What's the simplest way to get a dump of all memcached keys into a file?
• How do I view the data in memcache?

Answer 3

Base on @mu 無 answer here. I've written a cache dump script.

The script dumps all the content of a memcached server. It's tested with Ubuntu 12.04 and a localhost memcached, so your milage may vary.

#!/usr/bin/env bash

echo 'stats items'  \
| nc localhost 11211  \
| grep -oe ':[0-9]*:'  \
| grep -oe '[0-9]*'  \
| sort  \
| uniq  \
| xargs -L1 -I{} bash -c 'echo "stats cachedump {} 1000" | nc localhost 11211'

What it does, it goes through all the cache slabs and print 1000 entries of each.

Please be aware of certain limits of this script i.e. it may not scale for a 5GB cache server for example. But it's useful for debugging purposes on a local machine.

Answer 4

If you have PHP & PHP-memcached installed, you can run

$ php -r '$c = new Memcached(); $c->addServer("localhost", 11211); var_dump( $c->getAllKeys() );'

Answer 5

Bash

To get list of keys in Bash, follow the these steps.

First, define the following wrapper function to make it simple to use (copy and paste into shell):

function memcmd() {
  exec {memcache}<>/dev/tcp/localhost/11211
  printf "%s\n%s\n" "$*" quit >&${memcache}
  cat <&${memcache}
}

---

Memcached 1.4.31 and above

You can use lru_crawler metadump all command to dump (most of) the metadata for (all of) the items in the cache.

As opposed to cachedump, it does not cause severe performance problems and has no limits on the amount of keys that can be dumped.

Example command by using the previously defined function:

memcmd lru_crawler metadump all

See: ReleaseNotes1431.

---

Memcached 1.4.30 and below

Get list of slabs by using items statistics command, e.g.:

memcmd stats items

For each slub class, you can get list of items by specifying slub id along with limit number (0 - unlimited):

memcmd stats cachedump 1 0
memcmd stats cachedump 2 0
memcmd stats cachedump 3 0
memcmd stats cachedump 4 0
...

^{Note: You need to do this for each memcached server.}

To list all the keys from all stubs, here is the one-liner (per one server):

for id in $(memcmd stats items | grep -o ":[0-9]\+:" | tr -d : | sort -nu); do
    memcmd stats cachedump $id 0
done

^{Note: The above command could cause severe performance problems while accessing the items, so it's not advised to run on live.}

---

Notes:

stats cachedump only dumps the HOT_LRU (IIRC?), which is managed by a background thread as activity happens. This means under a new enough version which the 2Q algo enabled, you'll get snapshot views of what's in just one of the LRU's.

If you want to view everything, lru_crawler metadump 1 (or lru_crawler metadump all) is the new mostly-officially-supported method that will asynchronously dump as many keys as you want. you'll get them out of order but it hits all LRU's, and unless you're deleting/replacing items multiple runs should yield the same results.

Source: GH-405.

---

Related:

• List all objects in memcached
• Writing a Redis client in pure bash (it's Redis, but very similar approach)
• Check other available commands at https://memcached.org/wiki
• Check out the protocol.txt docs file.

Answer 6

The easiest way is to use python-memcached-stats package, https://github.com/abstatic/python-memcached-stats

The keys() method should get you going.

Example -

from memcached_stats import MemcachedStats
mem = MemcachedStats()

mem.keys()
['key-1',
 'key-2',
 'key-3',
 ... ]

Answer 7

I was using Java's spyMemcached, and used this code. It is based on Anshul Goyal's answer

@Autowired
@Qualifier("initMemcachedClient")
private MemcachedClient memcachedClient;

public List<String> getCachedKeys(){
    Set<Integer> slabIds = new HashSet<>();
    Map<SocketAddress, Map<String, String>> stats;
    List<String> keyNames = new ArrayList<>();

    // Gets all the slab IDs
    stats = memcachedClient.getStats("items");
    stats.forEach((socketAddress, value) -> {
        System.out.println("Socket address: "+socketAddress.toString());
        value.forEach((propertyName, propertyValue) -> {
            slabIds.add(Integer.parseInt(propertyName.split(":")[1]));
        });
    });

    // Gets all keys in each slab ID and adds in List keyNames
    slabIds.forEach(slabId -> {
        Map<SocketAddress, Map<String, String>> keyStats = memcachedClient.getStats("cachedump "+slabId+" 0");
        keyStats.forEach((socketAddress, value) -> {
            value.forEach((propertyName, propertyValue) -> {
                keyNames.add(propertyName);
            });
        });
    });

    System.out.println("number of keys: "+keyNames.size());
    return keyNames;
}

Answer 8

Java Solution:

Thanks! @Satvik Nema Your solution helped me to find the approach, but it doesn't work for memcached 2.4.6 version. (implementation 'com.googlecode.xmemcached:xmemcached:2.4.6') Not sure when did new method getStatsByItem included. I figured out required changes using documentation and below code worked for me.

// Gets all the slab IDs
            Set<Integer> slabIds = new HashSet<>();
            Map<InetSocketAddress, Map<String, String>> itemsMap = null;
            try {
                itemsMap = this.memcachedClient.getStatsByItem("items");
            } catch (Exception e) {
                log.error("Failed while pulling 'items'. ERROR", e);
            }
    
            if (Objects.nonNull(itemsMap)) {
                itemsMap.forEach((key, value) -> {
                    log.info("itemsMap {} : {}", key, value);
                    value.forEach((k, v) -> {
                        slabIds.add(Integer.parseInt(k.split(":")[1]));
                    });
                });
            }
    
            // Gets all keys in each slab ID and adds in List keyNames
            slabIds.forEach(slabId -> {
                Map<InetSocketAddress, Map<String, String>> keyStats = null;
                try {
                    keyStats = this.memcachedClient.getStatsByItem("cachedump " + slabId + " 0");
                } catch (Exception e) {
                    log.error("Failed while pulling 'cachedump' for slabId: {}. ERROR", slabId, e);
                }
                if (Objects.nonNull(keyStats)) {
                    keyStats.forEach((socketAddress, value) -> {
                        value.forEach((propertyName, propertyValue) -> {
                            //keyNames.add(propertyName);
                            log.info("keyName: {} Value: {}", propertyName, propertyValue);
                        });
                    });
                }
            });