Merge branch 'master' into staging

2 files changed, 432 insertions, 40 deletions

diff --git a/doc/guix-cookbook.texi b/doc/guix-cookbook.texi
index 66f94a0fe7..1b081a820e 100644
--- a/doc/guix-cookbook.texi
+++ b/doc/guix-cookbook.texi
@@ -58,6 +58,7 @@ Translation Project}.
 * Scheme tutorials::            Meet your new favorite language!
 * Packaging::                   Packaging tutorials
 * System Configuration::        Customizing the GNU System
+* Advanced package management:: Power to the users!
 
 * Acknowledgments::             Thanks!
 * GNU Free Documentation License::  The license of this document.
@@ -124,14 +125,14 @@ and @code{#f} stand for the booleans "true" and "false", respectively.
 
 Examples of valid expressions:
 
-@example scheme
+@lisp
 > "Hello World!"
 "Hello World!"
 > 17
 17
 > (display (string-append "Hello " "Guix" "\n"))
 "Hello Guix!"
-@end example
+@end lisp
 
 @item
 This last example is a function call nested in another function call.  When a
@@ -142,66 +143,66 @@ last evaluated expression as its return value.
 @item
 Anonymous functions are declared with the @code{lambda} term:
 
-@example scheme
+@lisp
 > (lambda (x) (* x x))
 #<procedure 120e348 at <unknown port>:24:0 (x)>
-@end example
+@end lisp
 
 The above procedure returns the square of its argument.  Since everything is
 an expression, the @code{lambda} expression returns an anonymous procedure,
 which can in turn be applied to an argument:
 
-@example scheme
+@lisp
 > ((lambda (x) (* x x)) 3)
 9
-@end example
+@end lisp
 
 @item
 Anything can be assigned a global name with @code{define}:
 
-@example scheme
+@lisp
 > (define a 3)
 > (define square (lambda (x) (* x x)))
 > (square a)
 9
-@end example
+@end lisp
 
 @item
 Procedures can be defined more concisely with the following syntax:
 
-@example scheme
+@lisp
 (define (square x) (* x x))
-@end example
+@end lisp
 
 @item
 A list structure can be created with the @code{list} procedure:
 
-@example scheme
+@lisp
 > (list 2 a 5 7)
 (2 3 5 7)
-@end example
+@end lisp
 
 @item
 The @emph{quote} disables evaluation of a parenthesized expression: the first
 term is not called over the other terms.  Thus it effectively returns a list
 of terms.
 
-@example scheme
+@lisp
 > '(display (string-append "Hello " "Guix" "\n"))
 (display (string-append "Hello " "Guix" "\n"))
 > '(2 a 5 7)
 (2 a 5 7)
-@end example
+@end lisp
 
 @item
 The @emph{quasiquote} disables evaluation of a parenthesized expression until
 a comma re-enables it.  Thus it provides us with fine-grained control over
 what is evaluated and what is not.
 
-@example scheme
+@lisp
 > `(2 a 5 7 (2 ,a 5 ,(+ a 4)))
 (2 a 5 7 (2 3 5 7))
-@end example
+@end lisp
 
 Note that the above result is a list of mixed elements: numbers, symbols (here
 @code{a}) and the last element is a list itself.
@@ -209,7 +210,7 @@ Note that the above result is a list of mixed elements: numbers, symbols (here
 @item
 Multiple variables can be named locally with @code{let}:
 
-@example scheme
+@lisp
 > (define x 10)
 > (let ((x 2)
         (y 3))
@@ -219,17 +220,17 @@ Multiple variables can be named locally with @code{let}:
 10
 > y
 ERROR: In procedure module-lookup: Unbound variable: y
-@end example
+@end lisp
 
 Use @code{let*} to allow later variable declarations to refer to earlier
 definitions.
 
-@example scheme
+@lisp
 > (let* ((x 2)
          (y (* x 3)))
     (list x y))
 (2 6)
-@end example
+@end lisp
 
 @item
 The keyword syntax is @code{#:}; it is used to create unique identifiers.
@@ -243,12 +244,12 @@ Scheme treats @code{%} exactly the same as any other letter.
 @item
 Modules are created with @code{define-module}.  For instance
 
-@example scheme
+@lisp
 (define-module (guix build-system ruby)
   #:use-module (guix store)
   #:export (ruby-build
             ruby-build-system))
-@end example
+@end lisp
 
 defines the module @code{guix build-system ruby} which must be located in
 @file{guix/build-system/ruby.scm} somewhere in the Guile load path.  It
@@ -342,7 +343,7 @@ install}).  Guix already provides a package definition which is a perfect
 example to start with.  You can look up its declaration with @code{guix edit
 hello} from the command line.  Let's see how it looks:
 
-@example scheme
+@lisp
 (define-public hello
   (package
     (name "hello")
@@ -362,7 +363,7 @@ serves as an example of standard GNU coding practices.  As such, it supports
 command-line arguments, multiple languages, and so on.")
     (home-page "https://www.gnu.org/software/hello/")
     (license gpl3+)))
-@end example
+@end lisp
 
 As you can see, most of it is rather straightforward.  But let's review the
 fields together:
@@ -422,7 +423,7 @@ setup later; for now we will go the simplest route.
 
 Save the following to a file @file{my-hello.scm}.
 
-@example scheme
+@lisp
 (use-modules (guix packages)
              (guix download)
              (guix build-system gnu)
@@ -446,7 +447,7 @@ serves as an example of standard GNU coding practices.  As such, it supports
 command-line arguments, multiple languages, and so on.")
   (home-page "https://www.gnu.org/software/hello/")
   (license gpl3+))
-@end example
+@end lisp
 
 We will explain the extra code in a moment.
 
@@ -563,7 +564,7 @@ nature of how the package definition is written.
 The @code{linux-libre} kernel package definition is actually a procedure which
 creates a package.
 
-@example scheme
+@lisp
 (define* (make-linux-libre version hash supported-systems
                            #:key
                            ;; A function that takes an arch and a variant.
@@ -574,19 +575,19 @@ creates a package.
                            (extra-options %default-extra-linux-options)
                            (patches (list %boot-logo-patch)))
   ...)
-@end example
+@end lisp
 
 The current @code{linux-libre} package is for the 5.1.x series, and is
 declared like this:
 
-@example scheme
+@lisp
 (define-public linux-libre
   (make-linux-libre %linux-libre-version
                     %linux-libre-hash
                     '("x86_64-linux" "i686-linux" "armhf-linux" "aarch64-linux")
                     #:patches %linux-libre-5.1-patches
                     #:configuration-file kernel-config))
-@end example
+@end lisp
 
 Any keys which are not assigned values inherit their default value from the
 @code{make-linux-libre} definition.  When comparing the two snippets above,
@@ -602,7 +603,7 @@ including an actual @file{.config} file as a native input to our custom
 kernel.  The following is a snippet from the custom @code{'configure} phase of
 the @code{make-linux-libre} package definition:
 
-@example scheme
+@lisp
 (let ((build  (assoc-ref %standard-phases 'build))
       (config (assoc-ref (or native-inputs inputs) "kconfig")))
 
@@ -613,13 +614,13 @@ the @code{make-linux-libre} package definition:
         (copy-file config ".config")
         (chmod ".config" #o666))
       (invoke "make" ,defconfig))
-@end example
+@end lisp
 
 Below is a sample kernel package.  The @code{linux-libre} package is nothing
 special and can be inherited from and have its fields overridden like any
 other package:
 
-@example scheme
+@lisp
 (define-public linux-libre/E2140
   (package
     (inherit linux-libre)
@@ -627,7 +628,7 @@ other package:
      `(("kconfig" ,(local-file "E2140.config"))
       ,@@(alist-delete "kconfig"
                       (package-native-inputs linux-libre))))))
-@end example
+@end lisp
 
 In the same directory as the file defining @code{linux-libre-E2140} is a file
 named @file{E2140.config}, which is an actual kernel configuration file.  The
@@ -640,7 +641,7 @@ The second way to create a custom kernel is to pass a new value to the
 @code{extra-options} keyword works with another function defined right below
 it:
 
-@example scheme
+@lisp
 (define %default-extra-linux-options
   `(;; https://lists.gnu.org/archive/html/guix-devel/2014-04/msg00039.html
    ("CONFIG_DEVPTS_MULTIPLE_INSTANCES" . #t)
@@ -666,11 +667,11 @@ it:
                        (string-append option "=n")))
                     options)
                "\n"))
-@end example
+@end lisp
 
 And in the custom configure script from the `make-linux-libre` package:
 
-@example scheme
+@lisp
 ;; Appending works even when the option wasn't in the
 ;; file.  The last one prevails if duplicated.
 (let ((port (open-file ".config" "a"))
@@ -679,13 +680,13 @@ And in the custom configure script from the `make-linux-libre` package:
   (close-port port))
 
 (invoke "make" "oldconfig"))))
-@end example
+@end lisp
 
 So by not providing a configuration-file the @file{.config} starts blank, and
 then we write into it the collection of flags that we want.  Here's another
 custom kernel:
 
-@example scheme
+@lisp
 (define %macbook41-full-config
   (append %macbook41-config-options
           %filesystems
@@ -702,7 +703,7 @@ custom kernel:
                       #:extra-version "macbook41"
                       #:patches (@@@@ (gnu packages linux) %linux-libre-5.1-patches)
                       #:extra-options %macbook41-config-options))
-@end example
+@end lisp
 
 In the above example @code{%filesystems} is a collection of flags enabling
 different filesystem support, @code{%efi-support} enables EFI support and
@@ -779,6 +780,394 @@ kernel, since certain modules which are expected to be built may not be
 available for inclusion into the initrd.
 
 @c *********************************************************************
+@node Advanced package management
+@chapter Advanced package management
+
+Guix is a functional package manager that offers many features beyond
+what more traditional package managers can do.  To the uninitiated,
+those features might not have obvious use cases at first.  The purpose
+of this chapter is to demonstrate some advanced package management
+concepts.
+
+@pxref{Package Management,,, guix, GNU Guix Reference Manual} for a complete
+reference.
+
+@menu
+* Guix Profiles in Practice::     Strategies for multiple profiles and manifests.
+@end menu
+
+@node Guix Profiles in Practice
+@section Guix Profiles in Practice
+
+Guix provides a very useful feature that may be quite foreign to newcomers:
+@emph{profiles}.  They are a way to group package installations together and all users
+on a same system are free to use as many profiles as they want.
+
+Whether you're a developer or not, you may find that multiple profiles bring you
+great power and flexibility.  While they shift the paradigm somewhat compared to
+@emph{traditional package managers}, they are very convenient to use once you've
+understood how to set them up.
+
+If you are familiar with Python's @samp{virtualenv}, you can think of a profile as a
+kind of universal @samp{virtualenv} that can hold any kind of software whatsoever, not
+just Python software.  Furthermore, profiles are self-sufficient: they capture
+all the runtime dependencies which guarantees that all programs within a profile
+will always work at any point in time.
+
+Multiple profiles have many benefits:
+
+@itemize
+@item
+Clean semantic separation of the various packages a user needs for different contexts.
+
+@item
+Multiple profiles can be made available into the environment either on login
+or within a dedicated shell.
+
+@item
+Profiles can be loaded on demand.  For instance, the user can use multiple
+shells, each of them running different profiles.
+
+@item
+Isolation: Programs from one profile will not use programs from the other, and
+they user can even install different versions of the same programs to the two
+profiles without conflict.
+
+@item
+Deduplication: Profiles share dependencies that happens to be the exact same.
+This makes multiple profiles storage-efficient.
+
+@item
+Reproducible: when used with declarative manifests, a profile can be fully
+specified by the Guix commit that was active when it was set up.  This means
+that the exact same profile can be @uref{https://guix.gnu.org/blog/2018/multi-dimensional-transactions-and-rollbacks-oh-my/, set up anywhere, anytime}, with just the
+commit information.  See the section on @ref{Reproducible profiles}.
+
+@item
+Easier upgrades and maintenance: Multiple profiles make it easy to keep
+package listings at hand and make upgrades completely friction-less.
+@end itemize
+
+Concretely, here follows some typical profiles:
+
+@itemize
+@item
+The dependencies of a project you are working on.
+
+@item
+Your favourite programming language libraries.
+
+@item
+Laptop-specific programs (like @samp{powertop}) that you don't need on a desktop.
+
+@item
+@TeX{}live (this one can be really useful when you need to install just one
+package for this one document you've just received over email).
+
+@item
+Games.
+@end itemize
+
+Let's dive in the set up!
+
+@node Basic setup with manifests
+@subsection Basic setup with manifests
+
+A Guix profile can be set up @emph{via} a so-called @emph{manifest specification} that looks like
+this:
+
+@lisp
+(specifications->manifest
+  '("package-1"
+    ;; Version 1.3 of package-2.
+    "package-2@@1.3"
+    ;; The "lib" output of package-3.
+    "package-3:lib"
+    ; ...
+    "package-N"))
+@end lisp
+
+@pxref{Invoking guix package,,, guix, GNU Guix Reference Manual}, for
+the syntax details.
+
+We can create a manifest specification per profile and install them this way:
+
+@example
+GUIX_EXTRA_PROFILES=$HOME/.guix-extra-profiles
+mkdir -p "$GUIX_EXTRA_PROFILES"/my-project # if it does not exist yet
+guix package --manifest=/path/to/guix-my-project-manifest.scm --profile="$GUIX_EXTRA_PROFILES"/my-project/my-project
+@end example
+
+Here we set an arbitrary variable @samp{GUIX_EXTRA_PROFILES} to point to the directory
+where we will store our profiles in the rest of this article.
+
+Placing all your profiles in a single directory, with each profile getting its
+own sub-directory, is somewhat cleaner.  This way, each sub-directory will
+contain all the symlinks for precisely one profile.  Besides, "looping over
+profiles" becomes obvious from any programming language (e.g. a shell script) by
+simply looping over the sub-directories of @samp{$GUIX_EXTRA_PROFILES}.
+
+Note that it's also possible to loop over the output of
+
+@example
+guix package --list-profiles
+@end example
+
+although you'll probably have to filter out @samp{~/.config/guix/current}.
+
+To enable all profiles on login, add this to your @samp{~/.bash_profile} (or similar):
+
+@example
+for i in $GUIX_EXTRA_PROFILES/*; do
+  profile=$i/$(basename "$i")
+  if [ -f "$profile"/etc/profile ]; then
+    GUIX_PROFILE="$profile"
+    . "$GUIX_PROFILE"/etc/profile
+  fi
+  unset profile
+done
+@end example
+
+Note to Guix System users: the above reflects how your default profile
+@samp{~/.guix-profile} is activated from @samp{/etc/profile}, that latter being loaded by
+@samp{~/.bashrc} by default.
+
+You can obviously choose to only enable a subset of them:
+
+@example
+for i in "$GUIX_EXTRA_PROFILES"/my-project-1 "$GUIX_EXTRA_PROFILES"/my-project-2; do
+  profile=$i/$(basename "$i")
+  if [ -f "$profile"/etc/profile ]; then
+    GUIX_PROFILE="$profile"
+    . "$GUIX_PROFILE"/etc/profile
+  fi
+  unset profile
+done
+@end example
+
+When a profile is off, it's straightforward to enable it for an individual shell
+without "polluting" the rest of the user session:
+
+@example
+GUIX_PROFILE="path/to/my-project" ; . "$GUIX_PROFILE"/etc/profile
+@end example
+
+The key to enabling a profile is to @emph{source} its @samp{etc/profile} file.  This file
+contains shell code that exports the right environment variables necessary to
+activate the software contained in the profile.  It is built automatically by
+Guix and meant to be sourced.
+It contains the same variables you would get if you ran:
+
+@example
+guix package --search-paths=prefix --profile=$my_profile"
+@end example
+
+Once again, see (@pxref{Invoking guix package,,, guix, GNU Guix Reference Manual})
+for the command line options.
+
+To upgrade a profile, simply install the manifest again:
+
+@example
+guix package -m /path/to/guix-my-project-manifest.scm -p "$GUIX_EXTRA_PROFILES"/my-project/my-project
+@end example
+
+To upgrade all profiles, it's easy enough to loop over them.  For instance,
+assuming your manifest specifications are stored in
+@samp{~/.guix-manifests/guix-$profile-manifest.scm}, with @samp{$profile} being the name
+of the profile (e.g. "project1"), you could do the following in Bourne shell:
+
+@example
+for profile in "$GUIX_EXTRA_PROFILES"/*; do
+  guix package --profile="$profile" --manifest="$HOME/.guix-manifests/guix-$profile-manifest.scm"
+done
+@end example
+
+Each profile has its own generations:
+
+@example
+guix package -p "$GUIX_EXTRA_PROFILES"/my-project/my-project --list-generations
+@end example
+
+You can roll-back to any generation of a given profile:
+
+@example
+guix package -p "$GUIX_EXTRA_PROFILES"/my-project/my-project --switch-generations=17
+@end example
+
+@node Required packages
+@subsection Required packages
+
+Activating a profile essentially boils down to exporting a bunch of
+environmental variables.  This is the role of the @samp{etc/profile} within the
+profile.
+
+@emph{Note: Only the environmental variables of the packages that consume them will
+be set.}
+
+For instance, @samp{MANPATH} won't be set if there is no consumer application for man
+pages within the profile.  So if you need to transparently access man pages once
+the profile is loaded, you've got two options:
+
+@itemize
+@item
+Either export the variable manually, e.g.
+@example
+export MANPATH=/path/to/profile$@{MANPATH:+:@}$MANPATH"
+@end example
+
+@item
+Or include @samp{man-db} to the profile manifest.
+@end itemize
+
+The same is true for @samp{INFOPATH} (you can install @samp{info-reader}),
+@samp{PKG_CONFIG_PATH} (install @samp{pkg-config}), etc.
+
+@node Default profile
+@subsection Default profile
+
+What about the default profile that Guix keeps in @samp{~/.guix-profile}?
+
+You can assign it the role you want.  Typically you would install the manifest
+of the packages you want to use all the time.
+
+Alternatively, you could keep it "manifest-less" for throw-away packages
+that you would just use for a couple of days.
+This way makes it convenient to run
+
+@example
+guix install package-foo
+guix upgrade package-bar
+@end example
+
+without having to specify the path to a profile.
+
+@node The benefits of manifests
+@subsection The benefits of manifests
+
+Manifests are a convenient way to keep your package lists around and, say,
+to synchronize them across multiple machines using a version control system.
+
+A common complaint about manifests is that they can be slow to install when they
+contain large number of packages.  This is especially cumbersome when you just
+want get an upgrade for one package within a big manifest.
+
+This is one more reason to use multiple profiles, which happen to be just
+perfect to break down manifests into multiple sets of semantically connected
+packages.  Using multiple, small profiles provides more flexibility and
+usability.
+
+Manifests come with multiple benefits.  In particular, they ease maintenance:
+
+@itemize
+@item
+When a profile is set up from a manifest, the manifest itself is
+self-sufficient to keep a "package listing" around and reinstall the profile
+later or on a different system.  For ad-hoc profiles, we would need to
+generate a manifest specification manually and maintain the package versions
+for the packages that don't use the default version.
+
+@item
+@code{guix package --upgrade} always tries to update the packages that have
+propagated inputs, even if there is nothing to do.  Guix manifests remove this
+problem.
+
+@item
+When partially upgrading a profile, conflicts may arise (due to diverging
+dependencies between the updated and the non-updated packages) and they can be
+annoying to resolve manually.  Manifests remove this problem altogether since
+all packages are always upgraded at once.
+
+@item
+As mentioned above, manifests allow for reproducible profiles, while the
+imperative @code{guix install}, @code{guix upgrade}, etc. do not, since they produce
+different profiles every time even when they hold the same packages.  See
+@uref{https://issues.guix.gnu.org/issue/33285, the related discussion on the matter}.
+
+@item
+Manifest specifications are usable by other @samp{guix} commands.  For example, you
+can run @code{guix weather -m manifest.scm} to see how many substitutes are
+available, which can help you decide whether you want to try upgrading today
+or wait a while.  Another example: you can run @code{guix pack -m manifest.scm} to
+create a pack containing all the packages in the manifest (and their
+transitive references).
+
+@item
+Finally, manifests have a Scheme representation, the @samp{<manifest>} record type.
+They can be manipulated in Scheme and passed to the various Guix @uref{https://en.wikipedia.org/wiki/Api, APIs}.
+@end itemize
+
+It's important to understand that while manifests can be used to declare
+profiles, they are not strictly equivalent: profiles have the side effect that
+they "pin" packages in the store, which prevents them from being
+garbage-collected (@pxref{Invoking guix gc,,, guix, GNU Guix Reference Manual})
+and ensures that they will still be available at any point in
+the future.
+
+Let's take an example:
+
+@enumerate
+@item
+We have an environment for hacking on a project for which there isn't a Guix
+package yet.  We build the environment using a manifest, and then run @code{guix
+   environment -m manifest.scm}.  So far so good.
+
+@item
+Many weeks pass and we have run a couple of @code{guix pull} in the mean time.
+Maybe a dependency from our manifest has been updated; or we may have run
+@code{guix gc} and some packages needed by our manifest have been
+garbage-collected.
+
+@item
+Eventually, we set to work on that project again, so we run @code{guix environment
+   -m manifest.scm}.  But now we have to wait for Guix to build and install
+stuff!
+@end enumerate
+
+Ideally, we could spare the rebuild time.  And indeed we can, all we need is to
+install the manifest to a profile and use @code{GUIX_PROFILE=/the/profile;
+. "$GUIX_PROFILE"/etc/profile} as explained above: this guarantees that our
+hacking environment will be available at all times.
+
+@emph{Security warning:} While keeping old profiles around can be convenient, keep in
+mind that outdated packages may not have received the latest security fixes.
+
+@node Reproducible profiles
+@subsection Reproducible profiles
+
+To reproduce a profile bit-for-bit, we need two pieces of information:
+
+@itemize
+@item
+a manifest,
+@item
+a Guix channel specification.
+@end itemize
+
+Indeed, manifests alone might not be enough: different Guix versions (or
+different channels) can produce different outputs for a given manifest.
+
+You can output the Guix channel specification with @samp{guix describe
+--format=channels}.
+Save this to a file, say @samp{channel-specs.scm}.
+
+On another computer, you can use the channel specification file and the manifest
+to reproduce the exact same profile:
+
+@example
+GUIX_EXTRA_PROFILES=$HOME/.guix-extra-profiles
+GUIX_EXTRA=$HOME/.guix-extra
+
+mkdir "$GUIX_EXTRA"/my-project
+guix pull --channels=channel-specs.scm --profile "$GUIX_EXTRA/my-project/guix"
+
+mkdir -p "$GUIX_EXTRA_PROFILES/my-project"
+"$GUIX_EXTRA"/my-project/guix/bin/guix package --manifest=/path/to/guix-my-project-manifest.scm --profile="$GUIX_EXTRA_PROFILES"/my-project/my-project
+@end example
+
+It's safe to delete the Guix channel profile you've just installed with the
+channel specification, the project profile does not depend on it.
+
+@c *********************************************************************
 @node Acknowledgments
 @chapter Acknowledgments
 
diff --git a/doc/guix.texi b/doc/guix.texi
index 746561ed97..a934626e5a 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -13988,6 +13988,9 @@ When @code{auto-login?} is false, GDM presents a log-in screen.
 When @code{auto-login?} is true, GDM logs in directly as
 @code{default-user}.
 
+@item @code{debug?} (default: @code{#f})
+When true, GDM writes debug messages to its log.
+
 @item @code{gnome-shell-assets} (default: ...)
 List of GNOME Shell assets needed by GDM: icon theme, fonts, etc.