How to match until the last occurrence of a character in bash shell

Question

I am using curl and cut on a output like below.

var=$(curl https://avc.com/actuator/info | tr '"' '\n' | grep - | head -n1 | cut -d'-' -f -1, -3)

Varible var gets have two kinds of values (one at a time).

HIX_MAIN-7ae526629f6939f717165c526dad3b7f0819d85b
HIX-R1-1-3b5126629f67892110165c524gbc5d5g1808c9b5

I am actually trying to get everything until the last '-'. i.e HIX-MAIN or HIX-R1-1.

The command shown works fine to get HIX-R1-1.

But I figured this is the wrong way to do when I have something something like only 1 - in the variable; it is getting me the entire variable value (e.g. HIX_MAIN-7ae526629f6939f717165c526dad3b7f0819d85b).

How do I go about getting everything up to the last '-' into the variable var?

It'd be interesting to know what the actual content from `curl` looks like; I wouldn't be surprised if the whole `tr | grep | head | cut` pipeline could be replaced with built-in string manipulation (or, if the content is in a format such as JSON, a single invocation of a syntax-aware tool such as `jq`). — Charles Duffy, Aug 19 '15 at 00:28
the curl output looks something like this {"build.number":"10393","branch.name":"HIX_MAIN-7ae526629f6939f717165c526dad3b7f0819d85b","profile.name":"hix","build.timestamp":"2015-08-16"} — mo0206, Aug 19 '15 at 06:54
Then `jq -r '.["branch.name"]'` would be the easier (and more reliable) way to extract that one value, as it would continue to work even if the format changed slightly (for instance, in the face of key reordering, or a new key/value pair being added earlier in the file). — Charles Duffy, Aug 19 '15 at 13:57
With jq 1.5 compiled with the optional regex support, one can do even better: `jq -r '.["branch.name"] | match("^(.*)-[^-]+$") | .captures[0].string'` will also trim the latest dash-separated segment, thus doing *all* the work you want in native `jq`. — Charles Duffy, Aug 19 '15 at 16:18

score 19 · Accepted Answer · answered Aug 19 '15 at 00:18

This removes everything from the last - to the end:

sed 's/\(.*\)-.*/\1/'

As examples:

$ echo HIX_MAIN-7ae52 | sed 's/\(.*\)-.*/\1/'
HIX_MAIN
$ echo HIX-R1-1-3b5126629f67 | sed 's/\(.*\)-.*/\1/'
HIX-R1-1

How it works

The sed substitute command has the form s/old/new/ where old is a regular expression. In this case, the regex is $.*$-.*. This works because $.*$- is greedy: it will match everything up to the last -. Because of the escaped parens,$...$, everything before the last - will be saved in group 1 which we can refer to as \1. The final .* matches everything after the last -. Thus, as long as the line contains a -, this regex matches the whole line and the substitute command replaces the whole line with \1.

+1, also reminding any `sed` newcomers that "escaping" parens in `sed` has the opposite behavior you might be used to from other programming languages: Unescaped they match a `(` character, and escaped with a backslash they create a matching group. With `sed -r`, you get the more familiar behavior, creating a matching group _unless_ preceded by a backslash. — Jeff Bowman, Aug 19 '15 at 18:06

Jeff Bowman · Answer 2 · 2015-08-19T00:37:22.637

You can use bash string manipulation:

$ foo=a-b-c-def-ghi
$ echo "${foo%-*}"
a-b-c-def

The operators, # and % are on either side of $ on a QWERTY keyboard, which helps to remember how they modify the variable:

#pattern trims off the shortest prefix matching "pattern".
##pattern trims off the longest prefix matching "pattern".
%pattern trims off the shortest suffix matching "pattern".
%%pattern trims off the longest suffix matching "pattern".

where pattern matches the bash pattern matching rules, including ? (one character) and * (zero or more characters).

Here, we're trimming off the shortest suffix matching the pattern -*, so ${foo%-*} will get you what you want.

Of course, there are many ways to do this using awk or sed, possibly reusing the sed command you're already running. Variable manipulation, however, can be done natively in bash without launching another process.

I've used the "#-comes-before-%" mnemonic to remember which is prefix and which is suffix, but I never thought about them being the immediate neighbors of `$` before. Nice. — chepner, Aug 19 '15 at 01:35

higuaro · Answer 3 · 2015-08-19T00:23:30.297

3

You can reverse the string with rev, cut from the second field and then rev again:

rev <<< "$VARIABLE" | cut -d"-" -f2- | rev

For HIX-R1-1----3b5126629f67892110165c524gbc5d5g1808c9b5, prints:

HIX-R1-1---

edited Aug 19 '15 at 00:23

answered Aug 19 '15 at 00:16

higuaro

15,730
4
36
43

3

Using `rev` twice seems like a sledgehammer being used to crack a peanut. – Jonathan Leffler Aug 19 '15 at 00:29
Yes, indeed, but IMHO can be more readable compared against a pure `sed` solution – higuaro Aug 19 '15 at 00:38

Jonathan Leffler · Answer 4 · 2015-08-19T00:28:48.830

I think you should be using sed, at least after the tr:

var=$(curl https://avc.com/actuator/info | tr '"' '\n' | sed -n '/-/{s/-[^-]*$//;p;q}')

The -n means "don't print by default". The /-/ looks for a line containing a dash; it then executes s/-[^-]*$// to delete the last dash and everything after it, followed by p to print and q to quit (so it only prints the first such line).

I'm assuming that the output from curl intrinsically contains multiple lines, some of them with unwanted double quotes in them, and that you need to match only the first line that contains a dash at all (which might very well not be the first line). Once you've whittled the input down to the sole interesting line, you could use pure shell techniques to get the result that's desired, but getting the sole interesting line is not as trivial as some of the answers seem to be assuming.

This approach nicely simplifies the OP's pipeline, eliminating both grep and head. +1. — John1024, Aug 19 '15 at 18:51

How to match until the last occurrence of a character in bash shell

4 Answers4

How it works

Linked