This TODO got added during our GHA refactoring a while ago, where I
thought it might make sense to run our itests in `bash`. We didn't do it
before the refactoring either. It seems good to keep them running in
`pwsh` instead, so lets just remove the TODO.
Investigation notes: Running in `bash` on Windows via `Git Bash` seems
like doing a lot of magic, especially with path handling. Things
[like](a02ed38e60/tests/installation/registry.ts (L143))
this break then, since `tar` even when using `Git Bash` [doesn't accept
a Windows
path](https://sourceforge.net/p/mingw/mailman/mingw-users/thread/54CE104A.7060108@hccnet.nl/).
Signed-off-by: Max Schmitt <max@schmitt.mx>
**Investigation:**
- According to
[here](https://github.com/microsoft/playwright/actions/runs/8351676198),
the job got cancelled, because someone force-pushed and another commit
took priority.
- We have `if: always()` for uploading the blobs, even if they are
cancelled. We expect, that the task before (`npx playwright test`)
finished writing the blob, but it didn't - cancelled in between. We
still upload - a broken one.
- We download the broken zip and it breaks from there on.
Proposed change: Instead of uploading always, upload if it did not get
cancelled.
Quoting from the GitHub Actions docs about `always()`:
> Warning: Avoid using always for any task that could suffer from a
critical failure, for example: getting sources, otherwise the workflow
may hang until it times out. If you want to run a job or step regardless
of its success or failure, use the recommended alternative: if: `${{
!cancelled() }}`
This is phase 1/2 where it changes our code to use this new condition.
The actual roll-out happens once it works for us.
Relates https://github.com/microsoft/playwright/issues/29451
`actions/upload-artifact@v4` comes with the following [breaking
change](https://github.com/actions/upload-artifact?tab=readme-ov-file#breaking-changes):
"Due to how Artifacts are created in this new version, it is no longer
possible to upload to the same named Artifact multiple times. You must
either split the uploads into multiple Artifacts with different names,
or only upload once. Otherwise you will encounter an error."
Due to that we cannot copy multiple blob report folders into the same
artifact name and rely on the action to merge them. Instead, as
suggested by their migration guide, we upload each blob report into a
uniquely named artifact with prefix `blob-report-` and then download all
of them into same directory.
This version change also affects how we store pull_request_number.txt
into an artifact. Previously we relied on the fact that uploading
artifact with the same name would silently override existing one, but
now it's an error. To overcome that, we upload PR number file into
uniquely named artifacts `pull-request-*` and later extract them into
same location with `unzip -n` which will never override existing file,
so we end up with single `pull_request_number.txt`.
Reference #28800
Also:
- remove `blob-report` directory at the start;
- markdown's `report.md` next to package.json;
- use default location in playwright's workflows.
References #24451.
* Fix report downloading from Azure (reports are now zipped)
* Extracted upload logic into an action
* Extracted PR number file generation into its own job
This reverts commit a1cdae6bff.
The problem with this approach is that each job overwrites the zip
artifact whereas previously it was merging all reports in the same
directory. We are going to zip .jsonl files instead.
Compressed size for `tests 1` blob report is 19Mb whil uncompressed one
is 211Mb. Also according to [GitHub
policy](https://docs.github.com/en/actions/using-workflows/storing-workflow-data-as-artifacts
) it is uncompressed size that is used for billing:
"Artifacts are uploaded during a workflow run, and you can view an
artifact's name and size in the UI. When an artifact is downloaded using
the GitHub UI, all files that were individually uploaded as part of the
artifact get zipped together into a single file. This means that billing
is calculated based on the size of the uploaded artifact and not the
size of the zip file."
For some reason pull_requests field on workflow_run is empty for pull
requests created from branches in forked repositories, see
https://github.com/orgs/community/discussions/25220. As a workaround we
store triggering pull request number in a file.
* Moved report merging and publishing logic into create_test_report.yml
shared between all workflows
* Merged reports are now published for try jobs on pull requests too. In
order to achieve that the logic had to be extracted into a separate
workflow triggered by
[workflow_run](https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#workflow_run),
this way it can access secrets even if the original workflow was not
able to.
* The blob report data flow is different depending on whether the
workflow is triggered by a pull request or a push:
- For `pull_request` the workflow doesn't have access to the secrets it
uploads the blob report to the GitHub
artifact storage. Later on the merge workflow uploads that blob report
to Azure blob storage.
- Workflows triggered by `push` event can read secrets. They upload blob
report directly to Azure blob storage
and the merge workflow downloads the report from there rather than from
GitHub artifacts.
`az storage blob download-batch` has been timing out over the last few
days, see upstream issue
https://github.com/Azure/azure-cli/issues/26567. Replacing it with a
simple bash script that discovers blobs with a given prefix and then
downloads one-by-one.
