about summary refs log tree commit diff homepage
path: root/blog/2020/gsoc/article/5.md
blob: 8658d9a8861fc54b11a2011c1c83cb1166131ce1 (plain) (blame)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
+++
title = "Sorting Things Out"
rss = "GSoC 2020: Sorting Things Out"
date = Date(2020, 8, 3)
tags = ["gsoc", "pip", "python"]
+++

Hi!  I really hope that everyone reading this is still doing okay,
and if that isn't the case, I wish you a good day!

## `pip` 20.2 Released!

Last Wednesday, `pip` 20.2 was released, delivering the `2020-resolver`
as well as many other improvements!  I was lucky to be able
to get the `fast-deps` feature to be included as part of the release.
A brief description of this *experimental* feature as well as testing
instruction can be found on [Python Discuss].

The public exposure of the feature also remind me of some further
{{pip 8681 optimization}} to make on {{pip 8670 "the lazy wheel"}}.
Hopefully without download parallelization it would not be too slow
to put off testing by concerned users of `pip`.

## Preparation for Download Parallelization

As of this moment, we already have:

* {{pip 8162#issuecomment-667504162 "Multithreading pool fallback working"}}
* An opt-in to use lazy wheel to optain dependency information,
  and thus getting a list of wheels at the end of resolution
  ready to be downloaded together

What's left is *only* to interject a parallel download somewhere after
the dependency resolution step.  Still, this struggles me way more than
I've ever imagined.  I got so stuck that I had to give myself a day off
in the middle of the week (and study some Rust), then I came up with
{{pip 8638 "something what was agreed upon as difficult to maintain"}}.

Indeed, a large part of this is my fault, for not communicating the design
thoroughly with `pip`'s maintainers and not carefully noting stuff down
during (verbal) discussions with my mentor.  Thankfully {{pip 8685
"Chris Hunt came to the rescue"}} and did a refactoring that will
make my future work much easier and cleaner.

[Python Discuss]: https://discuss.python.org/t/announcement-pip-20-2-release/4863/2