send-patch.txt

#+TITLE:     Submit patches
#+AUTHOR:    Fabrice Niessen
#+EMAIL:     fni@xxx.com
#+DESCRIPTION:
#+KEYWORDS:
#+LANGUAGE:  en
#+OPTIONS:   H:3 num:t toc:t ^:{}

#+PROPERTY:  eval     no
#+PROPERTY:  shebang  "#!/bin/bash\n"
#+SETUPFILE: bigblow.setup

* Commit guidelines

This list hopes to help all of us to improve our commits.

** Changes

- Make commits of logical units.

- Check for unnecessary whitespace before committing.

- Do not check in commented out code or unneeded files.

- Make sure that you have tests for the bug you are fixing.

- Make sure that the test suite passes after your commit.

- Do not attach your patch, but submit it "inline" in the mail body, /unless
  you cannot teach your mailer to leave the formatting of the patch alone/.

** Commit messages

[[http://chris.beams.io/posts/git-commit/][How to Write a Git Commit Message]]

- The first line of the commit message ("*subject line*") should be a *short
  description* (50 characters is the /soft/ limit).  It should be *capitalized* and
  it should *not end with a period*.

- The body should provide a meaningful commit message, which:

  + uses the active voice in the *imperative present tense*: "fix bug", not "fixed
    bug" or "fixes bug" -- we prefer to describe "what the patch does" rather
    than "what the patch did".

  + includes motivation for the change ("why"), and contrasts its implementation
    with previous behavior.

Keep in mind that commit messages are often read *without* seeing the diffs, and
without the possibility to see which other files you committed just before
that particular commit!

If the ChangeLog is a one-liner, you can use it as the commit summary.

Note for Emacs -- It is suggested to write the ChangeLog first and then use ~C-c
C-a~ from the ~*VC-Log*~ buffer.

** Release notes

We have introduced the possibility to insert release notes in log messages.  In
order to make them stand out, all such notes must be enclosed in
a ~#+begin_releasenotes~ .. ~#end_releasenotes~ block.

** Example

*** Commit message

By settling on an "official commit log format":

#+begin_example
TITLE

PARAGRAPH-1

PARAGRAPH-2

,* FILE (FUNC): DESCRIPTION-OF-CHANGE.
#+end_example

Example:

#+begin_example
Summarize changes in around 50 characters or less

More detailed explanatory text, if necessary, to present a bird-eye view that is
hard to grasp by looking at the diffs.  Wrap it to about 72 characters or so.
In some contexts, the first line is treated as the subject of an email and the
rest of the text as the body.  The blank line separating the summary from the
body is critical; some tools can get confused if you run the two together.

Explain the problem that this commit is solving. Focus on why you
are making this change as opposed to how (the code explains that).
Are there side effects or other unintuitive consequenses of this
change? Here's the place to explain them.

Further paragraphs come after blank lines.

- Bullet points are okay, too

- Typically a hyphen is used for the bullet, with blank lines in between, but
  conventions vary here

- Use a hanging indent

If you use an issue tracker, put references to them at the bottom, like this:

Resolves: #123
See also: #456, #789
#+end_example

*** ChangeLog

ChangeLog files could be generated from the commit messages:

#+begin_example
YYYY-MM-DD  AUTHOR  <EMAIL>

TITLE
,* FILE (FUNC): DESCRIPTION-OF-CHANGE.
#+end_example

(modulo some bol TABs).

Though, ChangeLog and commit logs may be orthogonal:

- A commit log describes /why/ (and /how/) code was changed.

- A ChangeLog helps you understand /where/ and /what/ code changed.

** TODO Real-life example                                             :noexport:

With a cover letter: http://patchwork.madduck.net/patch/1141/

* Code

** Send patch
   :PROPERTIES:
   :tangle:   bin/sendpatch
   :END:

This script sends patches:

- to the SVN repository (that is, /commits/ them)
- to mailing lists (that is, /emails/ them)

*** Version number

#+name: current-time
#+begin_src emacs-lisp :eval yes :results silent :tangle no
(format-time-string "%Y%m%d.%H%M")
#+end_src

#+begin_src shell :noweb yes
    cat << EOF
This is $(basename $0), version: <<current-time()>>.

EOF
#+end_src

*** Usage

#+begin_src shell
Usage ()
{
    cat << EOF 1>&2
Usage: $(basename $0) [OPTION]... [FILE]...
Submit patches per email.

  -h, --help     display this help and exit

Report $(basename $0) bugs to fni@xxx.com
Commit guidelines: <http://svn/mc/xxx/sendpatch/trunk/send-patch.html>
EOF
    exit 2
}

while true; do
    case "$1" in
        -h | --help) Usage ;;
        ,* ) break ;;
    esac
done
# now do something with $@
#+end_src

*** Configuration

**** Per project options

#+begin_src shell
mailto="mc-reviews@xxx.com"
prefix="[PATCH]"
excludefileregexp=""
#+end_src

**** Per user options

#+begin_src shell
patchfileprefix=""
commitmsgfileprefix=""
emailfileprefix=""
asksubject="yes"
autocommit="ask"
autosendemail="ask"
autodeletetmpfiles="ask"
emailcommand='blat "$email" -to "$mailto" -subject "$prefix $subject" -server "mail" -f "John Doe <johndoe@example.com>" -charset UTF-8'
#+end_src

A mailsend example with password input
#+begin_src sh
askemailpassword="yes"
emailcommand='cat $email | mailsend.exe -smtp mail.missioncriticalit.com -to "$mailto" -f mva@missioncriticalit.com -sub "$prefix $subject" -user mva@missioncriticalit.com -pass "$emailpassword" -v -auth -ssl -port 465'
#+end_src

**** Example

A commonly configured option could be:

#+begin_src shell :tangle no
excludefileregexp="(\\\.html|\\\.xml)"
#+end_src

#+begin_warning
Note the use of *3 backslashes*!
#+end_warning

Other command-line email agents:

#+begin_src shell :tangle no
emailcommand='sendemail -s mail -o tls=no -o message-charset=UTF-8 -f "John Doe <johndoe@example.com>" -t $mailto -u "$prefix $subject" -o message-file=$email'
#+end_src

**** Files

All the /default/ settings can be overridden in:

1. =~/.sendpatchrc= (global user config),
2. =sendpatch.common= (project config) in the root directory of a working copy, or
3. =sendpatch.params= (user config for the project) in the root directory of
   a working copy,

in that order.

Remember that those configuration files must be encoded with *Unix line
endings*!

*** Helper functions

The exit status follows the conventions for programs such as ~grep~, ~cmp~, and
~diff~:

#+begin_src shell
die () {
    printf "$(basename $0): $@\n" > /dev/stderr
    exit 2                              # an error occurred
}
#+end_src

#+begin_src shell
cecho () {
    black="\e[30m"
    red="\e[31m"
    green="\e[32m"
    yellow="\e[33m"
    blue="\e[34m"
    magenta="\e[35m"
    cyan="\e[36m"
    white="\e[37m"

    BLACK="\e[1;30m"
    RED="\e[1;31m"
    GREEN="\e[1;32m"
    YELLOW="\e[1;33m"
    BLUE="\e[1;34m"
    MAGENTA="\e[1;35m"
    CYAN="\e[1;36m"
    WHITE="\e[1;37m"

    reset="\e[0m"

    color=${YELLOW}
    printf "$color$1$reset\n"
}
#+end_src

**** VCS in use

#+begin_src shell
vcs_info () {
    local default="tfs"

    # recursively search for `.svn' directory
    while [[ ! ( -d .svn ) && ( "$(pwd)" != "/" ) ]]; do
        cd ..
    done

    if [[ -d .svn ]]; then
        printf "svn\n"
    else # not found
        printf "$default\n"
    fi
}
#+end_src

**** TFS

#+begin_src shell
fixtfsbinaryfiles () {
    sed -r "s/^(.*): files differ$/Index: \1\n===================================================================\nCannot display: file marked as a binary type.\n===================================================================/"
}
#+end_src

**** Yes/No prompt function

#+begin_src shell
ask () {
    while true; do
        if [ "${2:-}" = "Y" ]; then
            prompt="Y/n"
            default=Y
        elif [ "${2:-}" = "N" ]; then
            prompt="y/N"
            default=N
        else
            prompt="y/n"
            default=
        fi

        # ask the question (without the need to press RET)
        printf "$1 ($prompt): "
        read -n 1 -r REPLY
        printf "\n" # just a final linefeed

        # default?
        if [ -z "$REPLY" ]; then
            REPLY=$default
        fi

        # check if the reply is valid
        case "$REPLY" in
            y*|Y*) return 0 ;;
            n*|N*) return 1 ;;
        esac
    done
}
#+end_src

See [[https://gist.github.com/1965569][usage samples]] for defaults (when the user presses ~RET~ without giving an
answer).

See [[http://stackoverflow.com/questions/1885525/how-do-i-prompt-a-user-for-confirmation-in-bash-script][How do I prompt a user for confirmation in bash script?]] for the regexp
match operator "=~".

*** Init

#+begin_src shell
tmpfile="/tmp/$(basename $0).$$"
trap "rm -f $tmpfile" EXIT SIGHUP SIGINT SIGQUIT SIGTERM

userconfig=~/.sendpatchrc
projectconfig=sendpatch.common
userprojectconfig=sendpatch.params

export vcs=$(vcs_info)
printf "VC backend: $vcs\n"

case $vcs in
    svn)
        VCS_STATUS="svn status --ignore-externals"
        ;;
    tfs)
        VCS_STATUS="tf status"
        ;;
    *)
        die "VCS backend is not supported"
        ;;
esac
#+end_src

*** Check current directory

Note that, for TFS, we impose to find a =sendpatch.common= file in order to
conclude that we're at the root directory of a working copy, or not.

#+begin_src shell
case $vcs in
    svn)
        [[ -d .svn ]] \
            || die "\`$(pwd)' is not the root directory of a working copy"
        ;;
    tfs)
        [[ -f $projectconfig ]] \
            || die "\`$(pwd)' is not the root directory of a working copy"
        ;;
esac
#+end_src

*** Get URL of project

#+begin_src shell
case $vcs in
    svn)
        url=$(svn info | grep -E "^URL")/
        ;;
    tfs)
        # get collection and project
        tf workfold > $tmpfile
        url=$(cat $tmpfile | grep -E "^(Collection| \\$/)" | sed "/^ \\$/{s/:.*//}")
        ;;
esac
#+end_src

*** Override settings

#+begin_src shell
[[ -f $userconfig ]] && . $userconfig
[[ -f $projectconfig ]] && . $projectconfig
[[ -f $userprojectconfig ]] && . $userprojectconfig
#+end_src

Default sender of the email is John Doe.  If not configured otherwise, bail out:

#+begin_src shell
printf "$emailcommand" | grep "John Doe" \
    && die "Please fix the sender address in the email command"
#+end_src

Default editor is ~$EDITOR~.  If not found, set Emacs:

#+begin_src shell
: ${EDITOR:="emacsclient"}
which "${EDITOR%% *}" > /dev/null 2>&1 || die "editor not found"
#+end_src

*** Display changed files

Shows what files have been modified in your working copy, what files have been
changed in the repository since your last update, and which files, if any,
conflict with your modifications.  The ~-u~ option is what forces the command to
access the repository for this check.

#+begin_src shell
cecho "\nChanged files (to be committed now):"
case $vcs in
    svn)
        ${VCS_STATUS} -u $@ | grep -E -v "^\? " | tee $tmpfile
        cat $tmpfile | cut -c 1-9 | grep -E "^M       \*" \
            && die "a newer revision exists on the server"
        cat $tmpfile | cut -c 1-9 | grep -E "^C" && die "conflicted item modification"
        ;;
    tfs)
        export rootdirwin=$(cygpath -w -l "$(pwd)")
        # printf "Root directory (in Windows format) : $rootdirwin\n"
        ${VCS_STATUS} $@ > $tmpfile
        # No idea why, but we can't directly pipe the output of `tf status'
        # command
        cat $tmpfile | grep -F -i "$rootdirwin" | tee $tmpfile.out
        mv $tmpfile.out $tmpfile
        ;;
esac
#+end_src

*** Display ignored files

#+begin_src shell
case $vcs in
    svn)
        cecho "\nUntracked files:"
        ${VCS_STATUS} $@ | grep -E "^\? " | grep -v " 0.*-"
        ;;
    tfs)
        ;;
esac
#+end_src

Stop if no changes are to be committed.  Though, that may mean that you should
~add~ some of the files currently ignored; hence, it's good to make this check
after having sent some information to the user...

#+begin_src shell
[[ $(cat $tmpfile | wc -l) -gt 0 ]] \
    || die "no changes to send to the repository"
#+end_src

*** Ask for short log message

#+begin_src shell
if [[ "$asksubject" != "no" ]]; then
    printf "\nSubject (short log): "
    read subject
fi
#+end_src

*** Generate name for patch files

#+begin_src shell
oldcounter=$(ls ${patchfileprefix}0???-*.patch 2> /dev/null | sort --reverse \
    | head -n 1 | sed "s/^${patchfileprefix}00*//" | cut -d "-" -f 1)
newcounterpadded=$(printf "%04d" "$(( oldcounter + 1 ))")
filename="$newcounterpadded-$(printf "$subject\n" | tr '[:upper:]' '[:lower:]' | sed 's/[^a-z0-9]/-/g')"

patch=${patchfileprefix}${filename}.patch
commitmsg=${commitmsgfileprefix}${filename}.log
email=${emailfileprefix}${filename}.txt
#+end_src

*** Make patch file

The patch file generated only reports ~file.bin: files differ~.  The fix for
getting such names included in the log (list of modified files) is to replace
that one-sentence with a traditional ~Index:~ marker.

#+begin_src shell
makediff $@ > $patch.tmp
retval=$?

if [ $retval -eq 0 ]; then
    cat $patch.tmp | fixtfsbinaryfiles > $patch
    rm $patch.tmp
else
    rm $patch.tmp
    exit 2
fi
#+end_src

~excludefileregexp~ allows one to exclude useless or unreadable diffs from the
email contents.  That does not have any impact on the commit process itself...

#+begin_src shell
if [[ -n "$excludefileregexp" ]]; then
    cat $patch |\
    awk -v pattern="^Index: .*$excludefileregexp" \
        'BEGIN {deleteline = 0;}
         $0 ~ pattern {deleteline = 1; next;}
         // && deleteline == 0 {print $0;}
         /^Index: / && deleteline == 1 {print $0; deleteline = 0;}' \
    > $patch.excluded 2> /dev/null
else
    cp $patch $patch.excluded
fi
#+end_src

*** Edit log message

#+begin_src shell
# ask for commit message
cat <<EOF > $email
# Hey, Emacs! This is a -*- Diff -*- file
# Please enter the commit message for your changes.  Lines starting with '#'
# will be ignored...
# The FIRST LINE of the commit message should be a short description and should
# skip the full stop.  It will become the subject of the email.
EOF
case $vcs in
    svn)
        makelogparams="$*"
        ;;
    tfs)
        makelogparams="-f $patch"
        ;;
esac
{ printf "$subject\n"; makelog $makelogparams | sort | uniq; printf "\n---\n"; printf "$url\n\n"; \
  cat $patch.excluded; } >> $email
mv $patch.excluded $patch
printf "\nEdit commit log message and email to send to '$mailto'...\n"
$EDITOR $email
test $? -eq 0 || die "edit failed"

# remove comments from editable part
sed '0,/^---$/{/^#[^+].*$/d;}' < $email > $email.nocomment; mv $email.nocomment $email
#+end_src

*** Convert email contents into UTF-8

#+begin_src shell
# convert email into UTF-8
case "$(file --brief $email 2> /dev/null | sed 's/unified diff output, //')" in
    UTF-8*)
        ;;
    ASCII*)
        ;;
    ISO-8859*)
        printf "Character encoding of diff output: ISO-8859... Converting it to UTF-8...\n"
        cat $email | iconv -c -f ISO-8859-1 -t UTF-8 > $tmpfile
        mv $tmpfile $email
        ;;
    ,*)
        printf "Potential problems found in email file $email: $(file --brief $email)\n"
        ask "Continue?" || die "exiting..."
        ;;
esac
printf "Email:         $email\n"
#+end_src

*** Deduce log message for commit

#+begin_src shell
sed '/^---$/,$d' < $email > $commitmsg
#+end_src

*** Deduce subject line for email

#+begin_src shell
# get new subject for email (first line of log message)
head -n 1 $email > $tmpfile

# replace accented letters by their unaccented equivalent
case "$(file --brief $tmpfile 2> /dev/null)" in
    UTF-8*)
        subject="$(cat $tmpfile | iconv -c -f UTF-8 -t ISO-8859-1 | unaccent)"
        ;;
    ISO-8859*|ASCII*)
        subject="$(cat $tmpfile | unaccent)"
        ;;
    *)
        die "unknown character encoding: $(file --brief $tmpfile).  Did you put the subject as first line?"
        ;;
esac
#+end_src

*** Auto-commit changes

#+begin_src shell
printf "\n"
if [[ "$autocommit" = "ask" ]]; then
    ask "Commit?" && askcommit="yes" || askcommit="no"
fi

if [[ "$autocommit" = "yes" || "$askcommit" = "yes" ]]; then
    printf "Committing...\n"
    case $vcs in
        svn)
            svn commit --file $commitmsg $@
            ;;
        tfs)
            tf checkin /comment:@$commitmsg /noprompt $@
            ;;
    esac
    rc=$?
else
    printf "Nothing gets committed...\n"
    rc=0
fi
test $rc -eq 0 || die "commit failed"
#+end_src

*** Auto-send email

#+begin_src shell
printf "\n"
if [[ "$autosendemail" = "ask" ]]; then
    printf "Environment for sending email:\n"
    # print the email command
    printf "$emailcommand\n"
    [[ -f $userconfig ]] && grep "excludefileregexp" $userconfig
    [[ -f $projectconfig ]] && grep "excludefileregexp" $projectconfig
    [[ -f $userprojectconfig ]] && grep "excludefileregexp" $userprojectconfig
    ask "Send email?" && asksendemail="yes" || asksendemail="no"
fi

if [[ "$autosendemail" = "yes" || "$asksendemail" = "yes" ]]; then
    printf "Sending email...\n"
    eval "$emailcommand"
    rc=$?
else
    printf "Nothing gets sent...\n"
    rc=0
fi
test $rc -eq 0 || die "send email failed"
#+end_src

*** Auto-delete temporary files

#+begin_src shell
printf "\n"
if [[ "$autodeletetmpfiles" = "ask" ]]; then
    ask "Delete temp files?" && askdeletetmpfiles="yes" || askdeletetmpfiles="no"
fi

if [[ "$autodeletetmpfiles" = "yes" || "$askdeletetmpfiles" = "yes" ]]; then
    printf "Deleting temp files...\n"
    rm $patch $commitmsg $email
    rc=$?
else
    printf "Nothing gets deleted...\n"
    rc=0
fi
test $rc -eq 0 || die "delete temp files failed"

## sendpatch ends here
#+end_src

** Display list of files
   :PROPERTIES:
   :tangle:   bin/makelog
   :END:

Display a list of files to be committed, for the commit log message.

#+begin_src shell
# Usage: makelog [-f diff-file] [svn diff options]
#
# Make a log message for a change, including the header, and the relative
# pathnames of the changed files.  Output goes to stdout.
#
# If a you have a pre-existing diff file, use `-f diff-file' to use this file.

printf "\n"

{ if [ "$1" = "-f" ]
then
    cat $2
else
    svn diff -x -w --no-diff-deleted --show-copies-as-adds $* 2>&1
fi; } | sed '/^Index:/!d' | sed 's/^Index: /\t* /' | sed 's/$/:/'
#+end_src

With the option ~--show-copies-as-adds~, copied (and moved) files are listed.

Using ~sed '/^Index:/!d'~ (instead of ~egrep "^Index:"~) allows to filter the lines
we're interested in, even if the file contains binary characters (such as
non-breakable spaces).  On the other hand, ~grep~ would simply declare that
"Binary file matches".

** Display changes
   :PROPERTIES:
   :tangle:   bin/makediff
   :END:

Display the changes made to all target files, and display deleted and new
files as well.

With the option ~-x -w~, ~svn diff~ *shows only non-whitespace line changes* between
two revisions.

With the option ~--no-diff-deleted~, differences for deleted files are not
printed.

#+begin_src shell
# Usage: makediff [svn diff options]
#
# Make a diff file, ready for review.  It contains the diffs of all files,
# followed by the text of new files.  Output goes to stdout.

tmpfile="/tmp/$(basename $0).$$"
trap "rm -f $tmpfile" EXIT SIGHUP SIGINT SIGQUIT SIGTERM

case $vcs in
    svn)
        svn diff -x -w --no-diff-deleted $* > $tmpfile
        retval=$?
        ;;
    tfs)
        tf diff $* /format:Unified > $tmpfile
        retval=$?
        cat $tmpfile \
            | grep -E -i -v "^(edit|modifier|add|ajouter|renommer): " \
            | sed -r "s/^(File:|Fichier\o240:)/Index:/" > $tmpfile.out
        mv $tmpfile.out $tmpfile
        ;;
esac

if [ $retval -eq 0 ]; then
    cat $tmpfile
else
    exit 2
fi
#+end_src

** Ignore files from the command line
   :PROPERTIES:
   :tangle:   bin/svnignore
   :END:

Add a file (or a file glob pattern) to ignore without deleting the previous
value of the ~svn:ignore~ property... from the command line.

#+begin_src shell
# Usage: svnignore admin/notes/\*.tex

target="$1"
targetdirectory=$(dirname $target)
targetfilepattern=$(basename $target)

svnignores=$(svn propget svn:ignore "$targetdirectory")
svnignores="$svnignores"$'\n'"$targetfilepattern"
printf "Ignore '$targetfilepattern'\n"
svn propset svn:ignore "$svnignores" "$targetdirectory"
#+end_src

** Unaccent
   :PROPERTIES:
   :tangle:   bin/unaccent
   :END:

Remove accents, replacing accented letters by their unaccented equivalent.

#+begin_src shell :shebang "#!/bin/sed -f\n"
# Mappings between accented chars and ASCII chars.
s/\o240/ /g
s/\o300/A/g
s/\o301/A/g
s/\o302/A/g
s/\o304/A/g
s/\o310/E/g
s/\o311/E/g
s/\o312/E/g
s/\o313/E/g
s/\o315/I/g
s/\o316/I/g
s/\o323/O/g
s/\o324/O/g
s/\o325/O/g
s/\o326/O/g
s/\o331/U/g
s/\o332/U/g
s/\o333/U/g
s/\o334/U/g
s/\o340/a/g
s/\o341/a/g
s/\o342/a/g
s/\o344/a/g
s/\o347/c/g
s/\o350/e/g
s/\o351/e/g
s/\o352/e/g
s/\o353/e/g
s/\o355/i/g
s/\o356/i/g
s/\o357/i/g
s/\o363/o/g
s/\o364/o/g
s/\o365/o/g
s/\o366/o/g
s/\o371/u/g
s/\o372/u/g
s/\o373/u/g
s/\o374/u/g
#+end_src

** Install environment
   :PROPERTIES:
   :tangle:   bin/install-sendpatch-files
   :END:

Symlink the files.

#+begin_src shell :results silent
ln -f -s $(pwd)/sendpatch ~/bin/sendpatch
ln -f -s $(pwd)/makelog   ~/bin/makelog
ln -f -s $(pwd)/makediff  ~/bin/makediff
ln -f -s $(pwd)/svnignore ~/bin/svnignore
ln -f -s $(pwd)/unaccent  ~/bin/unaccent
#+end_src

#  LocalWords:  Fabrice Niessen inline SVN VCS TFS UTF Unaccent Symlink