Fix processing of unknown variable expansion types.

Our shell is (was) one of the last not to do this correctly.

Expansions are supposed to happen only when the command in which
they occur is being

Fix processing of unknown variable expansion types.

Our shell is (was) one of the last not to do this correctly.

Expansions are supposed to happen only when the command in which
they occur is being executed, not while it is being parsed.
If the expansion only happens them, errors should only be
detected then.

Make it work like that (I saw after I fixed this that FreeBSD
had done it, long ago, almost the same way - it is kind of an
obvious thing to do).

This will allow code like

if test it is shell X
then
commands using shell X specific expansion ops
else if it is shell Y
then
commands using shell Y specific expansion ops
else ...
fi

Previously expansion errors were detected while parsing, so
if we're not shell X, and don't implement something that it
does (some extension to the standard) that would have generated
a parser syntax error, and the script could not be executed
(despite the line with the error never being executed).

Note that this change does not handle all such possible
extensions, just this one. Others are much harder.

One side effect of this change is that sh will now continue
reading a variable expansion until it locates the terminating
'}' (in ${var} forms) regardless of how broken it obviously
is (to our shell) whereas previously it would have bailed out
as soon as an oddity was spotted.

show more ...

Implement expandvar() : runs var/arith/cmdsub expansions on var value

expandvar() is like expandenv() and expandstr() - each expands
variable values in different contexts, expandenv() when processin

Implement expandvar() : runs var/arith/cmdsub expansions on var value

expandvar() is like expandenv() and expandstr() - each expands
variable values in different contexts, expandenv() when processing
environment vars at startup (and there is no existing state to lose),
but is always using unsafe data, so never runs command substitutions,
expandstr() while in the middle of processing commands (mostly to
expand the prompt variables, so there is probably half a command
tree in the process of being built) - it uses the promptcmds option
to decide whether to run command substitutions. expandvar() is for
intermediate situations, where a variable is to be used during
processing. It will run command substitutions if the variable
value concerned was not imported from the environment.

There are currently no uses of expandvar(), so this change alters
nothing in the shell - but there will be in the near future.

show more ...

Delete a no-longer-used #define that referred to a struct field that
no longer exists. Also correct a couple of typos in comments. NFC.

PR standards/42829

Implement parameter and arithmetic expansion of $ENV
before using it as the name of a file from which to
read startup commands for the shell. This continues
to happen for all in

PR standards/42829

Implement parameter and arithmetic expansion of $ENV
before using it as the name of a file from which to
read startup commands for the shell. This continues
to happen for all interactive shells, and non-interactive
shells for which the posix option is not set (-o posix).

On any actual error, or if an attempt is made to use
command substitution, then the value of ENV is used
unchanged as the file name.

The expansion complies with POSIX XCU 2.5.3, though that
only requires parameter expansion - arithmetic expansion
is an extension (but for us, it is much easier to do, than
not to do, and it allows some weird stuff, if you're so
inclined....) Note that there is no ~ expansion (use $HOME).

show more ...

Revamp aliases - as dumb an idea as they are, if we're going
to have them, they should work as documented, not cause core
dumps, reference after free, incorrect replacements, failing
to implement ali

Revamp aliases - as dumb an idea as they are, if we're going
to have them, they should work as documented, not cause core
dumps, reference after free, incorrect replacements, failing
to implement alias after alias, ...

The big comment that ended:
This is a good idea ------- ***NOT***
and the hack it was describing are gone.

Note that most of this was from original CVS version 1.1
code (ie: came from the original import, even before 4.4-Lite
was merged. That is, May 1994. And no-one in 24.5 years
noticed (or at least complained about) all the bugs (or at
least, most of them)).

With these changes, aliases ought to work (if you can call it
that) as they are expected to by POSIX. Now if only we could
get POSIX to delete them (or make them optional)...

Changes partly inspired by similar changes made by FreeBSD,
(as was the previous change to alias.c, forgot ack in commit
log for that one, apologies) but done a little differently,
and perhaps with a slightly better outcome.

show more ...

NFC. Need a grain of const

Add support for $'...' quoting (based upon C "..." strings, with \ expansions.)

Implementation largely obtained from FreeBSD, with adaptations to meet the
needs and style of this sh, some updates to

Add support for $'...' quoting (based upon C "..." strings, with \ expansions.)

Implementation largely obtained from FreeBSD, with adaptations to meet the
needs and style of this sh, some updates to agree with the current POSIX spec,
and a few other minor changes.

The POSIX spec for this ( http://austingroupbugs.net/view.php?id=249 )
[see note 2809 for the current proposed text] is yet to be approved,
so might change. It currently leaves several aspects as unspecified,
this implementation handles those as:

Where more than 2 hex digits follow \x this implementation processes the
first two as hex, the following characters are processed as if the \x
sequence was not present. The value obtained from a \nnn octal sequence
is truncated to the low 8 bits (if a bigger value is written, eg: \456.)
Invalid escape sequences are errors. Invalid \u (or \U) code points are
errors if known to be invalid, otherwise can generate a '?' character.
Where any escape sequence generates nul ('\0') that char, and the rest of
the $'...' string is discarded, but anything remaining in the word is
processed, ie: aaa$'bbb\0ccc'ddd produces the same as aaa'bbb'ddd.

Differences from FreeBSD:
FreeBSD allows only exactly 4 or 8 hex digits for \u and \U (as does C,
but the current sh proposal differs.) reeBSD also continues consuming
as many hex digits as exist after \x (permitted by the spec, but insane),
and reject \u0000 as invalid). Some of this is possibly because that
their implementation is based upon an earlier proposal, perhaps note 590 -
though that has been updated several times.

Differences from the current POSIX proposal:
We currently always generate UTF-8 for the \u & \U escapes. We should
generate the equivalent character from the current locale's character set
(and UTF8 only if that is what the current locale uses.)
If anyone would like to correct that, go ahead.

We (and FreeBSD) generate (X & 0x1F) for \cX escapes where we should generate
the appropriate control character (SOH for \cA for example) with whatever
value that has in the current character set. Apart from EBCDIC, which
we do not support, I've never seen a case where they differ, so ...

show more ...

Implement PS1, PS2 and PS4 expansions (variable expansions, arithmetic
expansions, and if enabled by the promptcmds option, command substitutions.)

A better LINENO implementation. This version deletes (well, #if 0's out)
the LINENO hack, and uses the LINENO var for both ${LINENO} and $((LINENO)).
(Code to invert the LINENO hack when required,

A better LINENO implementation. This version deletes (well, #if 0's out)
the LINENO hack, and uses the LINENO var for both ${LINENO} and $((LINENO)).
(Code to invert the LINENO hack when required, like when de-compiling the
execution tree to provide the "jobs" command strings, is still included,
that can be deleted when the LINENO hack is completely removed - look for
refs to VSLINENO throughout the code. The var funclinno in parser.c can
also be removed, it is used only for the LINENO hack.)

This version produces accurate results: $((LINENO)) was made as accurate
as the LINENO hack made ${LINENO} which is very good. That's why the
LINENO hack is not yet completely removed, so it can be easily re-enabled.
If you can tell the difference when it is in use, or not in use, then
something has broken (or I managed to miss a case somewhere.)

The way that LINENO works is documented in its own (new) section in the
man page, so nothing more about that, or the new options, etc, here.

This version introduces the possibility of having a "reference" function
associated with a variable, which gets called whenever the value of the
variable is required (that's what implements LINENO). There is just
one function pointer however, so any particular variable gets at most
one of the set function (as used for PATH, etc) or the reference function.
The VFUNCREF bit in the var flags indicates which func the variable in
question uses (if any - the func ptr, as before, can be NULL).

I would not call the results of this perfect yet, but it is close.

show more ...

After discussions with Jilles Tjoelker (FreeBSD shell) and
following a suggestion from him, the way the fix to PR bin/50993
was implemented has changed a little. There are three steps involved
in p

After discussions with Jilles Tjoelker (FreeBSD shell) and
following a suggestion from him, the way the fix to PR bin/50993
was implemented has changed a little. There are three steps involved
in processing a here document, reading it, parsing it, and then
evaluating it before applying it to the correct file descriptor for
the command to use. The third of those is not related to this
problem, and has not changed. The bug was caused by combining the
first two steps into one (and not doing it correctly - which would be
hard that way.) The fix is to split the first two stages into
separate events. The original fix moved the 2nd stage (parsing)
to just immediately before the 3rd stage (evaluation.) Jilles
pointed out some unwanted side effects from doing it that way, and
suggested moving the 2nd stage to immediately after the first.
This commit makes that change. The effect is to revert the changes
to expand.c and parser.h (which are no longer needed) and simplify
slightly the change to parser.c. (from kre@)

show more ...

PR bin/50993 - this is a significant rewrite of the way that here
documents are processed. Now, when first detected, they are
simply read (the only change made to the text is to join lines
ended wit

PR bin/50993 - this is a significant rewrite of the way that here
documents are processed. Now, when first detected, they are
simply read (the only change made to the text is to join lines
ended with a \ to the subsequent line, otherwise end marker detection
does not work correctly (for here docs with an unquoted endmarker
only of course.) This patch also moves the "internal subroutine"
for looking for the end marker out of readtoken1() (which had to
happen as readtoken1 is no longer reading the here doc when it is
needed) - that uses code mostly taken from FreeBSD's sh (thanks!)
and along the way results in some restrictions on what the end
marker can be being removed. We still do not allow all we should.
(from kre@)

show more ...

PR bin/43469 - correctly handle quoting of the pattern part of ${var%pat}
type expansions. (from kre)

add crude $LINENO support for FreeBSD

Correctly apply IFS to unquoted text in ${x-text}.
Fixes PR/26058 and the 'for i in ${x-a b c}; do ...' and ${x-'a b' c}.
I can't find a PR for the latter problem.
Regression test goind in shortly.

Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22249, verified by myself.

Fixes from David Laight:
- ansification
- format of output of jobs command (etc)
- job identiers %+, %- etc
- $? and $(...)
- correct quoting of output of set, export -p and readonly -p
- differentia

Fixes from David Laight:
- ansification
- format of output of jobs command (etc)
- job identiers %+, %- etc
- $? and $(...)
- correct quoting of output of set, export -p and readonly -p
- differentiation between nornal and 'posix special' builtins
- correct behaviour (posix) for errors on builtins and special builtins
- builtin printf and kill
- set -o debug (if compiled with DEBUG)
- cd src obj (as ksh - too useful to do without)
- unset -e name, remove non-readonly variable from export list.
(so I could unset -e PS1 before running the test shell...)

show more ...

un-__P functions declared in parser.h. host programs include parser.h,
and so it shouldn't use __P. (this should probably be done better, by
not declaring the parser functions in headers used by ho

un-__P functions declared in parser.h. host programs include parser.h,
and so it shouldn't use __P. (this should probably be done better, by
not declaring the parser functions in headers used by host programs,
but this works well enough.)

show more ...

compile with WARNS = 2

Patches from Tor Egge (via Havard Eidnes) to fix various bugs in field
splitting and combining.
(Note: Some of this are not strictly bugs, but differences between traditional
Bourne shell and POSIX.)

PR/2808: Remove trailing whitespace (from FreeBSD)

Merge in my changes from vangogh, and fix the x=`false`; echo $? == 0
bug.

convert to new RCS id conventions.

I added the documented in the manual but not implemented variable expansions:

${#WORD}
${WORD%PAT}
${WORD%%PAT}
${WORD#PAT}
${WORD##PAT}

From Christos:
1. Fix `-' quoting in [ ] expressions.
2. Fix expansion of variables in redirections

Add RCS ids.