From 61263e1ba3a5d9b31da5876b1c5248cee3859996 Mon Sep 17 00:00:00 2001 From: Ludovic Courtès Date: Thu, 15 Oct 2020 18:48:30 +0200 Subject: doc: Add "Build Utilities" section. * doc/guix.texi (Build Utilities): New section. (Defining Packages): Refer to it. (Build Systems): Likewise. * doc/guix-cookbook.texi (Extended example): Likewise. --- doc/guix-cookbook.texi | 3 + doc/guix.texi | 276 ++++++++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 278 insertions(+), 1 deletion(-) diff --git a/doc/guix-cookbook.texi b/doc/guix-cookbook.texi index 9dffbca196..2845391db4 100644 --- a/doc/guix-cookbook.texi +++ b/doc/guix-cookbook.texi @@ -1143,6 +1143,9 @@ then restore the previous working directory. A ``@command{sed}-like'' function. @end table +@xref{Build Utilities,,, guix, GNU Guix Reference Manual}, for more +information on these utilities. + @subsubsection Module prefix The license in our last example needs a prefix: this is because of how the diff --git a/doc/guix.texi b/doc/guix.texi index c9c031ccdb..26de6790fe 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -252,6 +252,7 @@ Programming Interface * Package Modules:: Packages from the programmer's viewpoint. * Defining Packages:: Defining new packages. * Build Systems:: Specifying how packages are built. +* Build Utilities:: Helpers for your package definitions and more. * The Store:: Manipulating the package store. * Derivations:: Low-level interface to package derivations. * The Store Monad:: Purely functional interface to the store. @@ -6085,6 +6086,7 @@ package definitions. * Package Modules:: Packages from the programmer's viewpoint. * Defining Packages:: Defining new packages. * Build Systems:: Specifying how packages are built. +* Build Utilities:: Helpers for your package definitions and more. * The Store:: Manipulating the package store. * Derivations:: Low-level interface to package derivations. * The Store Monad:: Purely functional interface to the store. @@ -6241,6 +6243,10 @@ represents the familiar GNU Build System, where packages may be configured, built, and installed with the usual @code{./configure && make && make check && make install} command sequence. +When you start packaging non-trivial software, you may need tools to +manipulate those build phases, manipulate files, and so on. @xref{Build +Utilities}, for more on this. + @item The @code{arguments} field specifies options for the build system (@pxref{Build Systems}). Here it is interpreted by @@ -6816,7 +6822,8 @@ The list of phases used for a particular package can be changed with the @end example means that all the phases described above will be used, except the -@code{configure} phase. +@code{configure} phase. @xref{Build Utilities}, for more info on +@code{modify-phases} and build phases in general. In addition, this build system ensures that the ``standard'' environment for GNU packages is available. This includes tools such as GCC, libc, @@ -7646,6 +7653,273 @@ with @code{build-expression->derivation} (@pxref{Derivations, @code{build-expression->derivation}}). @end defvr +@node Build Utilities +@section Build Utilities + +As soon as you start writing non-trivial package definitions +(@pxref{Defining Packages}) or other build actions +(@pxref{G-Expressions}), you will likely start looking for helpers for +``shell-like'' actions---creating directories, copying and deleting +files recursively, manipulating build phases, and so on. The +@code{(guix build utils)} module provides such utility procedures. + +When writing package definitions, most build systems load @code{(guix +build utils)} (@pxref{Build Systems}). Thus, when writing custom build +phases or similar, you can usually assume those procedures are in scope. + +When writing g-expressions, you can import @code{(guix build utils)} on +the ``build side'' using @code{with-imported-modules} and then put it in +scope with the @code{use-modules} form (@pxref{Using Guile Modules,,, +guile, GNU Guile Reference Manual}): + +@lisp +(with-imported-modules '((guix build utils)) ;import it + (computed-file "empty-tree" + #~(begin + ;; Put it in scope. + (use-modules (guix build utils)) + + ;; Happily use its 'mkdir-p' procedure. + (mkdir-p (string-append #$output "/a/b/c"))))) +@end lisp + +The remainder of this section is the reference for most of the utility +procedures provided by @code{(guix build utils)}. + +@c TODO Document what's missing. + +@subsection Dealing with Store File Names + +This section of procedures deals with store file names. + +@deffn {Scheme Procedure} %store-directory +Return the directory name of the store. +@end deffn + +@deffn {Scheme Procedure} store-file-name? @var{file} +Return true if @var{file} is in the store. +@end deffn + +@deffn {Scheme Procedure} strip-store-file-name @var{file} +Strip the @file{/gnu/store} and hash from @var{file}, a store file name. +The result is typically a @code{"PACKAGE-VERSION"} string. +@end deffn + +@deffn {Scheme Procedure} package-name->name+version @var{name} +Given @var{name}, a package name like @code{"foo-0.9.1b"}, return two +values: @code{"foo"} and @code{"0.9.1b"}. When the version part is +unavailable, @var{name} and @code{#f} are returned. The first hyphen +followed by a digit is considered to introduce the version part. +@end deffn + +@subsection File Types + +The procedures below deal with files and file types. + +@deffn {Scheme Procedure} directory-exists? @var{dir} +Return @code{#t} if @var{dir} exists and is a directory. +@end deffn + +@deffn {Scheme Procedure} executable-file? @var{file} +Return @code{#t} if @var{file} exists and is executable. +@end deffn + +@deffn {Scheme Procedure} symbolic-link? @var{file} +Return @code{#t} if @var{file} is a symbolic link (aka. a ``symlink''). +@end deffn + +@deffn {Scheme Procedure} elf-file? @var{file} +@deffnx {Scheme Procedure} ar-file? @var{file} +@deffnx {Scheme Procedure} gzip-file? @var{file} +Return @code{#t} if @var{file} is, respectively, an ELF file, an +@code{ar} archive (such as a @file{.a} static library), or a gzip file. +@end deffn + +@deffn {Scheme Procedure} reset-gzip-timestamp @var{file} [#:keep-mtime? #t] +If @var{file} is a gzip file, reset its embedded timestamp (as with +@command{gzip --no-name}) and return true. Otherwise return @code{#f}. +When @var{keep-mtime?} is true, preserve @var{file}'s modification time. +@end deffn + +@subsection File Manipulation + +The following procedures and macros help create, modify, and delete +files. They provide functionality comparable to common shell utilities +such as @command{mkdir -p}, @command{cp -r}, @command{rm -r}, and +@command{sed}. They complement Guile's extensive, but low-level, file +system interface (@pxref{POSIX,,, guile, GNU Guile Reference Manual}). + +@deffn {Scheme Syntax} with-directory-excursion @var{directory} @var{body}@dots{} +Run @var{body} with @var{directory} as the process's current directory. + +Essentially, this macro changes the current directory to @var{directory} +before evaluating @var{body}, using @code{chdir} (@pxref{Processes,,, +guile, GNU Guile Reference Manual}). It changes back to the initial +directory when the dynamic extent of @var{body} is left, be it @i{via} +normal procedure return or @i{via} a non-local exit such as an +exception. +@end deffn + +@deffn {Scheme Procedure} mkdir-p @var{dir} +Create directory @var{dir} and all its ancestors. +@end deffn + +@deffn {Scheme Procedure} install-file @var{file} @var{directory} +Create @var{directory} if it does not exist and copy @var{file} in there +under the same name. +@end deffn + +@deffn {Scheme Procedure} make-file-writable @var{file} +Make @var{file} writable for its owner. +@end deffn + +@deffn {Scheme Procedure} copy-recursively @var{source} @var{destination} @ + [#:log (current-output-port)] [#:follow-symlinks? #f] [#:keep-mtime? #f] +Copy @var{source} directory to @var{destination}. Follow symlinks if +@var{follow-symlinks?} is true; otherwise, just preserve them. When +@var{keep-mtime?} is true, keep the modification time of the files in +@var{source} on those of @var{destination}. Write verbose output to the +@var{log} port. +@end deffn + +@deffn {Scheme Procedure} delete-file-recursively @var{dir} @ + [#:follow-mounts? #f] +Delete @var{dir} recursively, like @command{rm -rf}, without following +symlinks. Don't follow mount points either, unless @var{follow-mounts?} +is true. Report but ignore errors. +@end deffn + +@deffn {Scheme Syntax} substitute* @var{clause}@dots{} +Substitute @var{regexp} in @var{file} by the string returned by +@var{body}. @var{body} is evaluated with each @var{match-var} bound to +the corresponding positional regexp sub-expression. For example: + +@lisp +(substitute* file + (("hello") + "good morning\n") + (("foo([a-z]+)bar(.*)$" all letters end) + (string-append "baz" letter end))) +@end lisp + +Here, anytime a line of @var{file} contains @code{hello}, it is replaced +by @code{good morning}. Anytime a line of @var{file} matches the second +regexp, @var{all} is bound to the complete match, @var{letters} is bound +to the first sub-expression, and @var{end} is bound to the last one. + +When one of the @var{match-var} is @code{_}, no variable is bound to the +corresponding match substring. + +Alternatively, @var{file} may be a list of file names, in which case +they are all subject to the substitutions. + +Be careful about using @code{$} to match the end of a line; by itself it +won't match the terminating newline of a line. +@end deffn + +@subsection File Search + +@cindex file, searching +This section documents procedures to search and filter files. + +@deffn {Scheme Procedure} file-name-predicate @var{regexp} +Return a predicate that returns true when passed a file name whose base +name matches @var{regexp}. +@end deffn + +@deffn {Scheme Procedure} find-files @var{dir} [@var{pred}] @ + [#:stat lstat] [#:directories? #f] [#:fail-on-error? #f] +Return the lexicographically sorted list of files under @var{dir} for +which @var{pred} returns true. @var{pred} is passed two arguments: the +absolute file name, and its stat buffer; the default predicate always +returns true. @var{pred} can also be a regular expression, in which +case it is equivalent to @code{(file-name-predicate @var{pred})}. +@var{stat} is used to obtain file information; using @code{lstat} means +that symlinks are not followed. If @var{directories?} is true, then +directories will also be included. If @var{fail-on-error?} is true, +raise an exception upon error. +@end deffn + +Here are a few examples where we assume that the current directory is +the root of the Guix source tree: + +@lisp +;; List all the regular files in the current directory. +(find-files ".") +@result{} ("./.dir-locals.el" "./.gitignore" @dots{}) + +;; List all the .scm files under gnu/services. +(find-files "gnu/services" "\\.scm$") +@result{} ("gnu/services/admin.scm" "gnu/services/audio.scm" @dots{}) + +;; List ar files in the current directory. +(find-files "." (lambda (file stat) (ar-file? file))) +@result{} ("./libformat.a" "./libstore.a" @dots{}) +@end lisp + +@deffn {Scheme Procedure} which @var{program} +Return the complete file name for @var{program} as found in +@code{$PATH}, or @code{#f} if @var{program} could not be found. +@end deffn + +@subsection Build Phases + +@cindex build phases +The @code{(guix build utils)} also contains tools to manipulate +@dfn{build phases} as found in @code{gnu-build-system} and in fact most +build systems (@pxref{Build Systems}). Build phases are represented as +association lists or ``alists'' (@pxref{Association Lists,,, guile, GNU +Guile Reference Manual}) where each key is a symbol for the name of the +phase, and the associated value is a procedure that accepts an arbitrary +number of arguments. + +Guile core and the @code{(srfi srfi-1)} module both provide tools to +manipulate alists. The @code{(guix build utils)} module complements +those with tools written with build phases in mind. + +@cindex build phases, modifying +@deffn {Scheme Syntax} modify-phases @var{phases} @var{clause}@dots{} +Modify @var{phases} sequentially as per each @var{clause}, which may +have one of the following forms: + +@lisp +(delete @var{old-phase-name}) +(replace @var{old-phase-name} @var{new-phase}) +(add-before @var{old-phase-name} @var{new-phase-name} @var{new-phase}) +(add-after @var{old-phase-name} @var{new-phase-name} @var{new-phase}) +@end lisp + +Where every @var{phase-name} above is an expression evaluating to a +symbol, and @var{new-phase} an expression evaluating to a procedure. +@end deffn + +The example below is taken from the definition of the @code{grep} +package. It adds a phase to run after the @code{install} phase, called +@code{fix-egrep-and-fgrep}. That phase is a procedure (@code{lambda*} +is for anonymous procedures) that takes a @code{#:outputs} keyword +argument and ignores extra keyword arguments (@pxref{Optional +Arguments,,, guile, GNU Guile Reference Manual}, for more on +@code{lambda*} and optional and keyword arguments.) The phase uses +@code{substitute*} to modify the installed @file{egrep} and @file{fgrep} +scripts so that they refer to @code{grep} by its absolute file name: + +@lisp +(modify-phases %standard-phases + (add-after 'install 'fix-egrep-and-fgrep + ;; Patch 'egrep' and 'fgrep' to execute 'grep' via its + ;; absolute file name instead of searching for it in $PATH. + (lambda* (#:key outputs #:allow-other-keys) + (let* ((out (assoc-ref outputs "out")) + (bin (string-append out "/bin"))) + (substitute* (list (string-append bin "/egrep") + (string-append bin "/fgrep")) + (("^exec grep") + (string-append "exec " bin "/grep"))) + #t)))) +@end lisp + +@c TODO: Add more examples. + @node The Store @section The Store -- cgit 1.4.1