Move E2E tests to larger machines to see if flakyness decreases (#46960)

Summary:
This change move E2E tests to larger mac machines to see if we can reduce the flakyness.
Most of the time the flakyness is due to slowness in connecting to the simulator or slowness in Metro to provide the bundle to RNTester when we run the tests in Debug.
More power should speed-up those processes.

E2E tests took < 30 min in this diff
on Main, they took > 120 min

## Changelog:
[Internal] - Move the E2E tests to bigger machines

Pull Request resolved: https://github.com/facebook/react-native/pull/46960

Test Plan: GHA

Reviewed By: blakef

Differential Revision: D64237659

Pulled By: cipolleschi

fbshipit-source-id: c2458413c8de70c07fae8f2b5f202371c6293815
This commit is contained in:
Riccardo Cipolleschi 2024-10-11 05:10:10 -07:00 committed by Facebook GitHub Bot
parent 2c4567562c
commit d4edbcffbf
2 changed files with 13 additions and 5 deletions

View File

@ -44,6 +44,7 @@ runs:
run: |
cd ${{ inputs.working-directory }}
yarn start &
sleep 5 # to give metro time to load
- name: Run tests
id: run-tests
shell: bash
@ -68,8 +69,13 @@ runs:
echo "Launch the app"
xcrun simctl launch $UDID ${{ inputs.app-id }}
if [[ ${{ inputs.flavor }} == 'Debug' ]]; then
# To give the app time to warm the metro's cache
sleep 5
fi
echo "Running tests with Maestro"
export MAESTRO_DRIVER_STARTUP_TIMEOUT=1800000 # 30 min. CI is extremely slow
export MAESTRO_DRIVER_STARTUP_TIMEOUT=1200000 # 20 min. CI is extremely slow
# Add retries for flakyness
MAX_ATTEMPTS=5
@ -112,6 +118,8 @@ runs:
video_record_1.mov
video_record_2.mov
video_record_3.mov
video_record_4.mov
video_record_5.mov
report.xml
- name: Store Logs
if: failure() && steps.run-tests.outcome == 'failure'

View File

@ -183,8 +183,8 @@ jobs:
react-native-version: ${{ needs.prepare_hermes_workspace.outputs.react-native-version }}
test_e2e_ios_rntester:
if: ${{ github.ref == 'refs/heads/main' || contains(github.ref, 'stable') || inputs.run-e2e-tests }}
runs-on: macos-13
# if: ${{ github.ref == 'refs/heads/main' || contains(github.ref, 'stable') || inputs.run-e2e-tests }}
runs-on: macos-13-large
needs:
[build_apple_slices_hermes, prepare_hermes_workspace, build_hermes_macos]
env:
@ -221,8 +221,8 @@ jobs:
flavor: ${{ matrix.flavor }}
test_e2e_ios_templateapp:
if: ${{ github.ref == 'refs/heads/main' || contains(github.ref, 'stable') || inputs.run-e2e-tests }}
runs-on: macos-13
# if: ${{ github.ref == 'refs/heads/main' || contains(github.ref, 'stable') || inputs.run-e2e-tests }}
runs-on: macos-13-large
needs: build_npm_package
env:
HERMES_WS_DIR: /tmp/hermes