From 6d12c16299c1654a909c69d52bcb99b02cca0962 Mon Sep 17 00:00:00 2001 From: Ludovic Courtès Date: Thu, 14 Sep 2023 17:18:09 +0200 Subject: doc: Make “crash course” xref more visible. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * doc/guix.texi (Using the Configuration System): Move the “Do not panic” note right after the first example. Clarify wording. --- doc/guix.texi | 35 ++++++++++++++++++++--------------- 1 file changed, 20 insertions(+), 15 deletions(-) (limited to 'doc/guix.texi') diff --git a/doc/guix.texi b/doc/guix.texi index 339dcb2a41..7e42a7151c 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -16834,24 +16834,37 @@ instance to support new system services. The operating system is configured by providing an @code{operating-system} declaration in a file that can then be passed to the @command{guix system} command (@pxref{Invoking guix system}). A -simple setup, with the default system services, the default Linux-Libre -kernel, initial RAM disk, and boot loader looks like this: +simple setup, with the default Linux-Libre +kernel, initial RAM disk, and a couple of system services added to those +provided by default looks like this: @findex operating-system @lisp @include os-config-bare-bones.texi @end lisp -This example should be self-describing. Some of the fields defined +The configuration is declarative and hopefully mostly self-describing. +It is actually code in the Scheme programming language; the whole +@code{(operating-system @dots{})} expression produces a @dfn{record} +with a number of @dfn{fields}. +Some of the fields defined above, such as @code{host-name} and @code{bootloader}, are mandatory. Others, such as @code{packages} and @code{services}, can be omitted, in -which case they get a default value. +which case they get a default value. @xref{operating-system Reference}, +for details about all the available fields. -Below we discuss the effect of some of the most important fields -(@pxref{operating-system Reference}, for details about all the available -fields), and how to @dfn{instantiate} the operating system using +Below we discuss the effect of some of the most important fields, +and how to @dfn{instantiate} the operating system using @command{guix system}. +@quotation Do not panic +@cindex Scheme programming language, getting started +Intimidated by the Scheme language or curious about it? The Cookbook +has a short section to get started that explains the fundamentals, which +you will find helpful when hacking your configuration. @xref{A Scheme +Crash Course,,, guix-cookbook, GNU Guix Cookbook}. +@end quotation + @unnumberedsubsec Bootloader @cindex legacy boot, on Intel machines @@ -17025,14 +17038,6 @@ Alternatively, the @code{modify-services} macro can be used: (delete avahi-service-type)) @end lisp -@quotation Do not panic -@cindex Scheme programming language, getting started -Intimidated by the Scheme language or curious about it? The Cookbook -has a short section to get started that explains the fundamentals, which -you will find helpful when hacking your configuration. @xref{A Scheme -Crash Course,,, guix-cookbook, GNU Guix Cookbook}. -@end quotation - @unnumberedsubsec Instantiating the System Assuming the @code{operating-system} declaration -- cgit 1.4.1 From 82abf6ddadc6139148660440a064e60ae68f238e Mon Sep 17 00:00:00 2001 From: Christopher Baines Date: Sat, 26 Aug 2023 10:08:56 +0100 Subject: services: guix: Add bffe-service-type. This is intended to replace the functionality of the Guix Build Coordinator queue builds script, and also provide a web interface for build farms. * gnu/services/guix.scm (): New record type. (bffe-configuration, bffe-configuration?, bffe-configuration-package, bffe-configuration-user, bffe-configuration-group, bffe-configuration-arguments bffe-configuration-extra-environment-variables): New procedures. (bffe-service-type): New variable. * gnu/tests/guix.scm (%test-bffe): New variable. * doc/guix.texi (Guix Services): Document the new service. --- doc/guix.texi | 59 +++++++++++++++++++++++ gnu/services/guix.scm | 127 +++++++++++++++++++++++++++++++++++++++++++++++++- gnu/tests/guix.scm | 81 +++++++++++++++++++++++++++++++- 3 files changed, 265 insertions(+), 2 deletions(-) (limited to 'doc/guix.texi') diff --git a/doc/guix.texi b/doc/guix.texi index 7e42a7151c..46cc8e1b80 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -38088,6 +38088,65 @@ File name of the file system key for the target volume. @node Guix Services @subsection Guix Services +@subsubheading Build Farm Front-End (BFFE) +The @uref{https://git.cbaines.net/guix/bffe/,Build Farm Front-End} +assists with building Guix packages in bulk. It's responsible for +submitting builds and displaying the status of the build farm. + +@defvar bffe-service-type +Service type for the Build Farm Front-End. Its value must be a +@code{bffe-configuration} object. +@end defvar + +@deftp {Data Type} bffe-configuration +Data type representing the configuration of the Build Farm Front-End. + +@table @asis +@item @code{package} (default: @code{bffe}) +The Build Farm Front-End package to use. + +@item @code{user} (default: @code{"bffe"}) +The system user to run the service as. + +@item @code{group} (default: @code{"bffe"}) +The system group to run the service as. + +@item @code{arguments} +A list of arguments to the Build Farm Front-End. These are passed to +the @code{run-bffe-service} procedure when starting the service. + +For example, the following value directs the Build Farm Front-End to +submit builds for derivations available from @code{data.guix.gnu.org} to +the Build Coordinator instance assumed to be running on the same +machine. + +@example +(list + #:build + (list + (build-from-guix-data-service + (data-service-url "https://data.guix.gnu.org") + (build-coordinator-url "http://127.0.0.1:8746") + (branches '("master")) + (systems '("x86_64-linux" "i686-linux")) + (systems-and-targets + (map (lambda (target) + (cons "x86_64-linux" target)) + '("aarch64-linux-gnu" + "i586-pc-gnu"))) + (build-priority (const 0)))) + #:web-server-args + '(#:event-source "https://example.com" + #:controller-args + (#:title "example.com build farm"))) +@end example + +@item @code{extra-environment-variables} (default: @var{'()}) +Extra environment variables to set via the shepherd service. + +@end table +@end deftp + @subsubheading Guix Build Coordinator The @uref{https://git.cbaines.net/guix/build-coordinator/,Guix Build Coordinator} aids in distributing derivation builds among machines diff --git a/gnu/services/guix.scm b/gnu/services/guix.scm index 99b21f52d8..9b19a48cfd 100644 --- a/gnu/services/guix.scm +++ b/gnu/services/guix.scm @@ -140,7 +140,17 @@ nar-herder-cached-compression-configuration-type nar-herder-cached-compression-configuration-level nar-herder-cached-compression-configuration-directory - nar-herder-cached-compression-configuration-directory-max-size)) + nar-herder-cached-compression-configuration-directory-max-size + + bffe-configuration + bffe-configuration? + bffe-configuration-package + bffe-configuration-user + bffe-configuration-group + bffe-configuration-arguments + bffe-configuration-extra-environment-variables + + bffe-service-type)) ;;;; Commentary: ;;; @@ -1030,3 +1040,118 @@ ca-certificates.crt file in the system profile." nar-herder-account))) (description "Run a Nar Herder server."))) + + +;;; +;;; Build Farm Front-end (BFFE) +;;; + +(define-record-type* + bffe-configuration make-bffe-configuration + bffe-configuration? + (package bffe-configuration-package + (default bffe)) + (user bffe-configuration-user + (default "bffe")) + (group bffe-configuration-group + (default "bffe")) + (arguments bffe-configuration-arguments) + (extra-environment-variables + bffe-configuration-extra-environment-variables + (default '()))) + +(define (bffe-shepherd-services config) + (define bffe-package + (bffe-configuration-package config)) + + (define start-script + (program-file + "run-bffe" + (with-extensions (cons + bffe-package + ;; This is a poorly constructed Guile load path, + ;; since it contains things that aren't Guile + ;; libraries, but it means that the Guile + ;; libraries needed for BFFE don't need to be + ;; individually specified here. + (map second (package-transitive-propagated-inputs + bffe-package))) + #~(begin + (use-modules (bffe) + (bffe manage-builds)) + + (setvbuf (current-output-port) 'line) + (setvbuf (current-error-port) 'line) + + (simple-format #t "starting the bffe:\n ~A\n" + (current-filename)) + + (apply run-bffe-service + (append + (list #:pid-file "/var/run/bffe/pid") + #$(bffe-configuration-arguments config))))) + #:guile guile-3.0)) + + (match-record config + (package user group arguments extra-environment-variables) + + (list + (shepherd-service + (documentation "Build Farm Front-end") + (provision '(bffe)) + (requirement '(networking)) + (start #~(make-forkexec-constructor + (list #$start-script) + #:user #$user + #:group #$group + #:pid-file "/var/run/bffe/pid" + #:directory "/var/lib/bffe" + #:environment-variables + `(,(string-append + "GUIX_LOCPATH=" #$glibc-utf8-locales "/lib/locale") + "LC_ALL=en_US.utf8" + #$@extra-environment-variables) + #:log-file "/var/log/bffe/server.log")) + (stop #~(make-kill-destructor)))))) + +(define (bffe-activation config) + #~(begin + (use-modules (guix build utils)) + + (define %user + (getpw #$(bffe-configuration-user config))) + + (chmod "/var/lib/bffe" #o755) + + (mkdir-p "/var/log/bffe") + + ;; Allow writing the PID file + (mkdir-p "/var/run/bffe") + (chown "/var/run/bffe" (passwd:uid %user) (passwd:gid %user)))) + +(define (bffe-account config) + (match-record config + (user group) + (list (user-group + (name group) + (system? #t)) + (user-account + (name user) + (group group) + (system? #t) + (comment "BFFE user") + (home-directory "/var/lib/bffe") + (shell (file-append shadow "/sbin/nologin")))))) + +(define bffe-service-type + (service-type + (name 'bffe) + (extensions + (list (service-extension shepherd-root-service-type + bffe-shepherd-services) + (service-extension activation-service-type + bffe-activation) + (service-extension account-service-type + bffe-account))) + (description + "Run the Build Farm Front-end."))) diff --git a/gnu/tests/guix.scm b/gnu/tests/guix.scm index ad0980a10c..240ded4825 100644 --- a/gnu/tests/guix.scm +++ b/gnu/tests/guix.scm @@ -37,7 +37,8 @@ #:use-module (ice-9 match) #:export (%test-guix-build-coordinator %test-guix-data-service - %test-nar-herder)) + %test-nar-herder + %test-bffe)) ;;; ;;; Guix Build Coordinator @@ -325,3 +326,81 @@ host all all ::1/128 trust")))))) (name "nar-herder") (description "Connect to a running Nar Herder server.") (value (run-nar-herder-test)))) + + +;;; +;;; Build Farm Front-end +;;; + +(define %bffe-os + (simple-operating-system + (service dhcp-client-service-type) + (service guix-build-coordinator-service-type) + (service bffe-service-type + (bffe-configuration + (arguments + #~(list + #:web-server-args + '(#:port 8767 + #:controller-args + (#:title "Test title")))))))) + +(define (run-bffe-test) + (define os + (marionette-operating-system + %bffe-os + #:imported-modules '((gnu services herd) + (guix combinators)))) + + (define forwarded-port 8767) + + (define vm + (virtual-machine + (operating-system os) + (memory-size 1024) + (port-forwardings `((,forwarded-port . 8767))))) + + (define test + (with-imported-modules '((gnu build marionette)) + #~(begin + (use-modules (srfi srfi-11) (srfi srfi-64) + (gnu build marionette) + (web uri) + (web client) + (web response)) + + (define marionette + (make-marionette (list #$vm))) + + (test-runner-current (system-test-runner #$output)) + (test-begin "bffe") + + (test-assert "service running" + (marionette-eval + '(begin + (use-modules (gnu services herd)) + (match (start-service 'bffe) + (#f #f) + (('service response-parts ...) + (match (assq-ref response-parts 'running) + ((pid) (number? pid)))))) + marionette)) + + (test-equal "http-get" + 200 + (let-values + (((response text) + (http-get #$(simple-format + #f "http://localhost:~A/" forwarded-port) + #:decode-body? #t))) + (response-code response))) + + (test-end)))) + + (gexp->derivation "bffe-test" test)) + +(define %test-bffe + (system-test + (name "bffe") + (description "Connect to a running Build Farm Front-end.") + (value (run-bffe-test)))) -- cgit 1.4.1 From 04f71edb73205d0bb82404de28a70ae17b897429 Mon Sep 17 00:00:00 2001 From: Alexey Abramov Date: Tue, 25 Jul 2023 12:59:56 +0200 Subject: services: dhcp-client-configuration: Allow provision override. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * gnu/services/networking.scm ()[shepherd-provision]: New field. (dhcp-client-shepherd-service): Honor it. * doc/guix.texi (Networking Setup): Document it. Co-authored-by: Ludovic Courtès --- doc/guix.texi | 6 ++++++ gnu/services/networking.scm | 11 +++++++---- 2 files changed, 13 insertions(+), 4 deletions(-) (limited to 'doc/guix.texi') diff --git a/doc/guix.texi b/doc/guix.texi index 46cc8e1b80..983b471fd8 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -20461,10 +20461,16 @@ non-loopback interfaces that can be activated. Otherwise the DHCP client listens only on the specified interfaces. @item @code{shepherd-requirement} (default: @code{'()}) +@itemx @code{shepherd-provision} (default: @code{'(networking)}) This option can be used to provide a list of symbols naming Shepherd services that this service will depend on, such as @code{'wpa-supplicant} or @code{'iwd} if you require authenticated access for encrypted WiFi or Ethernet networks. + +Likewise, @code{shepherd-provision} is a list of Shepherd service names +(symbols) provided by this service. You might want to change the +default value if you intend to run several DHCP clients, only one of +which provides the @code{networking} Shepherd service. @end table @end deftp diff --git a/gnu/services/networking.scm b/gnu/services/networking.scm index e2f6e6c0ca..ec34137d39 100644 --- a/gnu/services/networking.scm +++ b/gnu/services/networking.scm @@ -89,6 +89,7 @@ dhcp-client-configuration? dhcp-client-configuration-package dhcp-client-configuration-interfaces + dhcp-client-configuration-shepherd-provision dhcp-client-configuration-shepherd-requirement dhcpd-service-type @@ -303,6 +304,8 @@ (default isc-dhcp)) (shepherd-requirement dhcp-client-configuration-shepherd-requirement (default '())) + (shepherd-provision dhcp-client-configuration-provision + (default '(networking))) (interfaces dhcp-client-configuration-interfaces (default 'all))) ;'all | list of strings @@ -310,19 +313,19 @@ (match-lambda ((? dhcp-client-configuration? config) (let ((package (dhcp-client-configuration-package config)) - (shepherd-requirement (dhcp-client-configuration-shepherd-requirement config)) + (requirement (dhcp-client-configuration-shepherd-requirement config)) + (provision (dhcp-client-configuration-shepherd-provision config)) (interfaces (dhcp-client-configuration-interfaces config)) (pid-file "/var/run/dhclient.pid")) (list (shepherd-service (documentation "Set up networking via DHCP.") - (requirement `(user-processes udev ,@shepherd-requirement)) + (requirement `(user-processes udev ,@requirement)) + (provision provision) ;; XXX: Running with '-nw' ("no wait") avoids blocking for a minute when ;; networking is unavailable, but also means that the interface is not up ;; yet when 'start' completes. To wait for the interface to be ready, one ;; should instead monitor udev events. - (provision '(networking)) - (start #~(lambda _ (define dhclient (string-append #$package "/sbin/dhclient")) -- cgit 1.4.1 From a839bb9777ac2686c41c48b7499123c7e8f9c281 Mon Sep 17 00:00:00 2001 From: Ludovic Courtès Date: Sun, 17 Sep 2023 17:34:17 +0200 Subject: doc: Fix typo. * doc/guix.texi (Specifying Channel Authorizations): Remove extra hyphen. --- doc/guix.texi | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc/guix.texi') diff --git a/doc/guix.texi b/doc/guix.texi index 983b471fd8..50c4984d71 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -6114,7 +6114,7 @@ the fingerprint of the OpenPGP used to sign it. @end enumerate Before pushing to your public Git repository, you can run @command{guix -git-authenticate} to verify that you did sign all the commits you are +git authenticate} to verify that you did sign all the commits you are about to push with an authorized key: @example -- cgit 1.4.1 From e5ed1712da049b1c3dcf01e0a7e02e48a8aff012 Mon Sep 17 00:00:00 2001 From: Mathieu Othacehe Date: Sat, 9 Sep 2023 17:57:25 +0200 Subject: image: Introduce the mbr-hybrid-raw image type. Until 209204e23b39af09e0ea92540b6fa00a60e6a0ae and d57cab764122af69d52d8cc9c843456044e5d7bc, the default image type used by "guix system image" was an MBR image with an ESP partition. Having both an MBR image and an ESP partition is handy because the image will boot on most x86 based systems using legacy BIOS and/or UEFI. We now have a distinction between MBR images and EFI images. Introduce a new MBR hybrid image type and default to it to restore the default behaviour. This also fixes the images section of (gnu ci) that was trying to install a BIOS bootloader on an EFI, GPT image and failing to do so. Signed-off-by: Mathieu Othacehe --- doc/guix.texi | 34 +++++++++++++++++++++++++++------- gnu/ci.scm | 2 +- gnu/system/image.scm | 14 ++++++++++++++ guix/scripts/system.scm | 2 +- 4 files changed, 43 insertions(+), 9 deletions(-) (limited to 'doc/guix.texi') diff --git a/doc/guix.texi b/doc/guix.texi index 50c4984d71..617b8463e3 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -40982,8 +40982,8 @@ QEMU monitor and the VM. @cindex image, creating disk images The @code{image} command can produce various image types. The image type can be selected using the @option{--image-type} option. It -defaults to @code{mbr-raw}. When its value is @code{iso9660}, the -@option{--label} option can be used to specify a volume ID with +defaults to @code{mbr-hybrid-raw}. When its value is @code{iso9660}, +the @option{--label} option can be used to specify a volume ID with @code{image}. By default, the root file system of a disk image is mounted non-volatile; the @option{--volatile} option can be provided to make it volatile instead. When using @code{image}, the bootloader @@ -41001,8 +41001,8 @@ qemu-system-x86_64 -enable-kvm -hda /tmp/my-image.qcow2 -m 1000 \ -bios $(guix build ovmf)/share/firmware/ovmf_x64.bin @end example -When using the @code{mbr-raw} image type, a raw disk image is produced; -it can be copied as is to a USB stick, for instance. Assuming +When using the @code{mbr-hybrid-raw} image type, a raw disk image is +produced; it can be copied as is to a USB stick, for instance. Assuming @code{/dev/sdc} is the device corresponding to a USB stick, one can copy the image to it using the following command: @@ -41139,7 +41139,7 @@ of the image. For the @code{image} action, create an image with given @var{type}. When this option is omitted, @command{guix system} uses the -@code{mbr-raw} image type. +@code{mbr-hybrid-raw} image type. @cindex ISO-9660 format @cindex CD image format @@ -45347,7 +45347,7 @@ then directly boot from it, without any kind of installation procedure. The @command{guix system image} command is able to turn an operating system definition into a bootable image. This command supports -different image types, such as @code{mbr-raw}, @code{iso9660} and +different image types, such as @code{mbr-hybrid-raw}, @code{iso9660} and @code{docker}. Any modern @code{x86_64} machine will probably be able to boot from an @code{iso9660} image. However, there are a few machines out there that require specific image types. Those machines, in general @@ -45611,8 +45611,24 @@ from them to simplify the @code{image} definition. The @code{(gnu system image)} module provides the following @code{image} definition variables. +@defvar mbr-disk-image +An MBR disk-image composed of a single ROOT partition. The ROOT +partition starts at a 1@tie{}MiB offset so that the bootloader can +install itself in the post-MBR gap. +@end defvar + +@defvar mbr-hybrid-disk-image +An MBR disk-image composed of two partitions: a 64 bits ESP partition +and a ROOT boot partition. The ESP partition starts at a 1@tie{}MiB +offset so that a BIOS compatible bootloader can install itself in the +post-MBR gap. The image can be used by @code{x86_64} and @code{i686} +machines supporting only legacy BIOS booting. The ESP partition ensures +that it can also be used by newer machines relying on UEFI booting, +hence the @emph{hybrid} denomination. +@end defvar + @defvar efi-disk-image -A MBR disk-image composed of two partitions: a 64 bits ESP partition and +A GPT disk-image composed of two partitions: a 64 bits ESP partition and a ROOT boot partition. This image can be used on most @code{x86_64} and @code{i686} machines, supporting BIOS or UEFI booting. @end defvar @@ -45703,6 +45719,10 @@ system image)} and the @code{(gnu system images @dots{})} modules. Build an image based on the @code{mbr-disk-image} image. @end defvar +@defvar mbr-hybrid-raw-image-type +Build an image based on the @code{mbr-hybrid-disk-image} image. +@end defvar + @defvar efi-raw-image-type Build an image based on the @code{efi-disk-image} image. @end defvar diff --git a/gnu/ci.scm b/gnu/ci.scm index 520ac28110..279dd4d910 100644 --- a/gnu/ci.scm +++ b/gnu/ci.scm @@ -268,7 +268,7 @@ otherwise use the IMAGE name." (if (member system %guix-system-supported-systems) `(,(image->job store (image - (inherit efi-disk-image) + (inherit mbr-hybrid-disk-image) (operating-system installation-os)) #:name "usb-image" #:system system) diff --git a/gnu/system/image.scm b/gnu/system/image.scm index 5b8da2f896..b1b928b222 100644 --- a/gnu/system/image.scm +++ b/gnu/system/image.scm @@ -77,6 +77,7 @@ root-partition mbr-disk-image + mbr-hybrid-disk-image efi-disk-image iso9660-image docker-image @@ -86,6 +87,7 @@ image-with-os mbr-raw-image-type + mbr-hybrid-raw-image-type efi-raw-image-type efi32-raw-image-type qcow2-image-type @@ -156,6 +158,13 @@ parent image record." (inherit root-partition) (offset root-offset)))))) +(define mbr-hybrid-disk-image + (image-without-os + (format 'disk-image) + (partition-table-type 'mbr) + (partitions + (list esp-partition root-partition)))) + (define efi-disk-image (image-without-os (format 'disk-image) @@ -217,6 +226,11 @@ set to the given OS." (name 'mbr-raw) (constructor (cut image-with-os mbr-disk-image <>)))) +(define mbr-hybrid-raw-image-type + (image-type + (name 'mbr-hybrid-raw) + (constructor (cut image-with-os mbr-hybrid-disk-image <>)))) + (define efi-raw-image-type (image-type (name 'efi-raw) diff --git a/guix/scripts/system.scm b/guix/scripts/system.scm index ec331809ef..547387d5e1 100644 --- a/guix/scripts/system.scm +++ b/guix/scripts/system.scm @@ -1169,7 +1169,7 @@ Some ACTIONS support additional ARGS.\n")) (debug . 0) (verbosity . #f) ;default (validate-reconfigure . ,ensure-forward-reconfigure) - (image-type . mbr-raw) + (image-type . mbr-hybrid-raw) (image-size . guess) (install-bootloader? . #t) (label . #f) -- cgit 1.4.1