Migrating to `use-package’—TIP 1: Do NOT use a naive `macroexpand’ to grok a `use-package’ declaration; use this wrapper instead

Foreword

The library use-package is very popular and its use is ubuiquitous. I would come across as very silly if I were to “introduce” use-package to the Emacs audience. So, I will skip the preliminaries and get straight to the meat of this article.

I am a pre–use-package Emacs user.

This means that I can get by without use-package.

This also means that when I come across a use-package declaration in the wild, I am baffled; I can instinctively understand what the different keywords–:mode, :hook, :init, :config, :preface: etc—mean, even if I cannot precisely state what they do.

A few months ago, I realized that use-package is part of Emacs 29.1. Now that use-package is formally “blessed”, I started migrating all my init snippets to a set of use-package declarations. After around a month of “on-and-off” effort, I have migrated most of my init snippets to use-package. Now, I proudly declare that there are no require-s—apart from the very “must have”-s—in my init file, and my Emacs starts up much quickly.

This post is a first among a series of posts where I document what a pre–use-package Emacs user MUST know in order to confidently and successfully migrate their configuration to use-package.

Intended Audience

The intended audience of this post are users

• who can confidently program in Emacs Lisp
• who are dithering on migrating to use-package either because of “I don’t need it” or “I don’t have time to read yet another obstruse GNU (?) Manual”
• who byte-compile their init file (Note this)
• who proclaim “Code is documentation; I don’t need documentation, show me the code!”

Objective

By the end of this post, you would know what form-s a given use-package call expands to.

Once you know that, you will be able to distribute your init logic among the various “sections” / “keywords” of use-package, and thereby make use of mechanics of use-package to simplify your init code.

What is use-package, and what does it expand to?

use-package is a fairly sophisticated macro.

By sophisticated, I mean, this:

An use-package invocation does NOT expand to the same set of forms under all circumstances.

That is, an use-package invocation expands to different form-s depending on the context or the environment in which it is expanded.

So, what factors in the environment control the behaviour of use-package?

As far I know, there are three:

1. use-package-expand-minimally: This is a user option, and is documented.

   If non-nil, make the expanded code as minimal as possible. This disables:

   • Printing to the *Messages* buffer of slowly-evaluating forms
   • Capturing of load errors (normally redisplayed as warnings)
   • Conditional loading of packages (load failures become errors)

   The main advantage to this variable is that, if you know your configuration works, it will make the byte-compiled file as minimal as possible. It can also help with reading macro-expanded definitions, to understand the main intent of what’s happening.

2. byte-compile-current-file: The dependence on this variable is not documented.

   (The intention of this post is to explore what role this variable plays in user-package invocation.)

   When this variable is ON—this variable is ON, when a file that uses use-package declarations is being byte-compiled—use-package emits code that contains eval-when-compile and eval-and-compile form-s.

   What does this mean?

   In the examples cited later in the article, you will notice that

   • eval-when-compile surrounds load-line of the package you intend to use.

     This means that, when your Emacs is starting up, you are only configuring the package, and NOT load-ing it.

     This

     “Don’t unconditionally require libraries in your init file; load libraries only when you need their services”

     is one of the fundamental design goals of use-package.

   • the eval-and-compile surrounds declare-function.

     More generally, eval-and-compile surrounds your :define-s, :functions and whatever you may have included in the :preface section.

     You use these keywords to deal with warnings you see during compilation phase.

3. use-package-compute-statistics: This is a user option, and is documented.

   If non-nil, compute statistics concerned use-package declarations. View the statistical report using use-package-report. Note that if this option is enabled, you must require use-package in your user init file at loadup time, or you will see errors concerning undefined variables.

   This variable is used for profiling your init config, and it is intended for “one-off” use. So, we will not explore the effects of this variable in this post.

A helper for inspecting a use-package incantation

Add the following snippet to your .emacs and activate it.

This snippet adds an advice to pp-macroexpand-last-sexp. The advice checks what macro you are expanding, and if you are expanding a use-package declaration, it will query you for the enviornment in which you want the use-package to be expanded. Then it macroexpand-s, your use-package call in that environment. As stated earlier, the environment influences how a use-package call behaves.

A helper to macroexpand a use-package declaration

(use-package use-package
  :config
  (advice-add 'pp-macroexpand-last-sexp :around
              (defun pp-macroexpand-last-sexp--around
                  (orig-fun &rest orig-args)
                (pcase-let*
                    ((`(,arg)
                      orig-args)
                     (sexp (pp-last-sexp))
                     (env (append
                           (cond
                            ((eq 'use-package (car sexp))
                             `((use-package-expand-minimally ,(y-or-n-p "Minimal"))
                               (byte-compile-current-file ,(when (y-or-n-p "Byte compilation")
                                                             (current-buffer)))
                               (comment (format "
;; use-package-expand-minimally:         %S
;; byte-compile-current-file:            %S

"
                                                use-package-expand-minimally
                                                (null (null byte-compile-current-file))))))
                            (t
                             `((comment "")))))))
                  ;; (message "%S" env)
                  (eval `(let* ,env
                           (if ',arg
                               (save-excursion
                                 (insert "\n\n")
                                 (insert comment)
                                 (apply ',orig-fun ',orig-args))
                             (apply ',orig-fun ',orig-args))))))))

Bind pp-macroexpand-last-sexp to convenient set of keys to test drive the above helper

I am binding C-c C-c to pp-macroexpand-last-sexp here.

A convenient key binding for pp-macroexpand-last-sexp

(local-set-key (kbd "C-c C-c") #'pp-macroexpand-last-sexp)

A use-package declaration for test driving the above helper

This is the use-package snippet we will try to unravel. Copy this to your *scratch* buffer.

A sample use-package declaration

(use-package rainbow-mode
  :functions (rainbow-x-color-luminance)
  :config
  (rainbow-x-color-luminance "cyan"))

The myriad forms the above use-package declaration expands to

• Put your cursor at the end of above use-package snippet and do C-u C-c C-c (= C-u M-x pp-macroexpand-last-sexp RET).
• Answer few queries on how you want your environment setup

Once that is done you will see the expansion of above use-package snippet right below your cursor.

Repeat this step for different values of the enviornment. Note specifically the presence of eval-when-compile and eval-and-compile calls when byte-compile-current-file is non-NIL.

Down below you see various form-s an use-package declaration takes.

Myriad expansions of the above snippet

  
;; use-package-expand-minimally:    t
;; byte-compile-current-file:       t
(progn
  (eval-and-compile
    (declare-function rainbow-x-color-luminance #1="rainbow-mode")
    (eval-when-compile
      (with-demoted-errors "Cannot load rainbow-mode: %S"
        nil (unless (featurep 'rainbow-mode) (load #1# nil t)))))
  (require 'rainbow-mode nil nil) (rainbow-x-color-luminance "cyan") t)


;; use-package-expand-minimally:    t
;; byte-compile-current-file:       nil
(progn
  (require 'rainbow-mode nil nil) (rainbow-x-color-luminance "cyan") t)


;; use-package-expand-minimally:    nil
;; byte-compile-current-file:       t
(progn
  (eval-and-compile
    (declare-function rainbow-x-color-luminance #1="rainbow-mode")
    (eval-when-compile
      (with-demoted-errors "Cannot load rainbow-mode: %S"
        nil (unless (featurep 'rainbow-mode) (load #1# nil t)))))
  (defvar use-package--warning71
    #'(lambda (keyword err)
        (let
            ((msg
              (format "%s/%s: %s" 'rainbow-mode keyword
                      (error-message-string err))))
          (display-warning 'use-package msg :error))))
  (condition-case-unless-debug err
      (if (not (require 'rainbow-mode nil t))
          (display-warning 'use-package
                           (format "Cannot load %s" 'rainbow-mode)
                           :error)
        (condition-case-unless-debug err
            (progn (rainbow-x-color-luminance "cyan") t)
          (error (funcall use-package--warning71 :config . #2=(err)))))
    (error (funcall use-package--warning71 :catch . #2#))))


;; use-package-expand-minimally:    nil
;; byte-compile-current-file:       nil
(progn
  (defvar use-package--warning70
    #'(lambda (keyword err)
        (let
            ((msg
              (format "%s/%s: %s" 'rainbow-mode keyword
                      (error-message-string err))))
          (display-warning 'use-package msg :error))))
  (condition-case-unless-debug err
      (if (not (require 'rainbow-mode nil t))
          (display-warning 'use-package
                           (format "Cannot load %s" 'rainbow-mode)
                           :error)
        (condition-case-unless-debug err
            (progn (rainbow-x-color-luminance "cyan") t)
          (error (funcall use-package--warning70 :config . #1=(err)))))
    (error (funcall use-package--warning70 :catch . #1#))))

Conclusion

Don’t avoid switching to use-package.

use-package is no esoteric science when you have the macroexpand and the above helper at hand.

This post would have convinced you that you can get more mileage out of simply expanding a use-package at hand, than grokking the use-package docstring or its manual .

Once you switch to use-package you will realize that

1. every library has its own fixed place in your config (as opposed to its config being all over your init file)
2. you have nuked all the stray require-s you added (but forgot to remove)

(1) means your init file is well-organized.

(2) means that your Emacs loads up very fast.

If you are averse to using use-package, you can use the above helper to unravel an “off-the-shelf” use-package configuration, and plug in the pure-Emacs Lisp expansion to your init file. You can then march forth as if no such thing as use-package ever exists in this (your?) universe.