1#!/usr/bin/env python3 2 3# Copyright 2017 The Glslang Authors. All rights reserved. 4# Copyright (c) 2018-2023 Valve Corporation 5# Copyright (c) 2018-2023 LunarG, Inc. 6# Copyright (c) 2023-2023 RasterGrid Kft. 7# 8# Licensed under the Apache License, Version 2.0 (the "License"); 9# you may not use this file except in compliance with the License. 10# You may obtain a copy of the License at 11# 12# http://www.apache.org/licenses/LICENSE-2.0 13# 14# Unless required by applicable law or agreed to in writing, software 15# distributed under the License is distributed on an "AS IS" BASIS, 16# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 17# See the License for the specific language governing permissions and 18# limitations under the License. 19 20# This script was heavily leveraged from KhronosGroup/glslang 21# update_glslang_sources.py. 22"""update_deps.py 23 24Get and build dependent repositories using known-good commits. 25 26Purpose 27------- 28 29This program is intended to assist a developer of this repository 30(the "home" repository) by gathering and building the repositories that 31this home repository depend on. It also checks out each dependent 32repository at a "known-good" commit in order to provide stability in 33the dependent repositories. 34 35Known-Good JSON Database 36------------------------ 37 38This program expects to find a file named "known-good.json" in the 39same directory as the program file. This JSON file is tailored for 40the needs of the home repository by including its dependent repositories. 41 42Program Options 43--------------- 44 45See the help text (update_deps.py --help) for a complete list of options. 46 47Program Operation 48----------------- 49 50The program uses the user's current directory at the time of program 51invocation as the location for fetching and building the dependent 52repositories. The user can override this by using the "--dir" option. 53 54For example, a directory named "build" in the repository's root directory 55is a good place to put the dependent repositories because that directory 56is not tracked by Git. (See the .gitignore file.) The "external" directory 57may also be a suitable location. 58A user can issue: 59 60$ cd My-Repo 61$ mkdir build 62$ cd build 63$ ../scripts/update_deps.py 64 65or, to do the same thing, but using the --dir option: 66 67$ cd My-Repo 68$ mkdir build 69$ scripts/update_deps.py --dir=build 70 71With these commands, the "build" directory is considered the "top" 72directory where the program clones the dependent repositories. The 73JSON file configures the build and install working directories to be 74within this "top" directory. 75 76Note that the "dir" option can also specify an absolute path: 77 78$ cd My-Repo 79$ scripts/update_deps.py --dir=/tmp/deps 80 81The "top" dir is then /tmp/deps (Linux filesystem example) and is 82where this program will clone and build the dependent repositories. 83 84Helper CMake Config File 85------------------------ 86 87When the program finishes building the dependencies, it writes a file 88named "helper.cmake" to the "top" directory that contains CMake commands 89for setting CMake variables for locating the dependent repositories. 90This helper file can be used to set up the CMake build files for this 91"home" repository. 92 93A complete sequence might look like: 94 95$ git clone git@github.com:My-Group/My-Repo.git 96$ cd My-Repo 97$ mkdir build 98$ cd build 99$ ../scripts/update_deps.py 100$ cmake -C helper.cmake .. 101$ cmake --build . 102 103JSON File Schema 104---------------- 105 106There's no formal schema for the "known-good" JSON file, but here is 107a description of its elements. All elements are required except those 108marked as optional. Please see the "known_good.json" file for 109examples of all of these elements. 110 111- name 112 113The name of the dependent repository. This field can be referenced 114by the "deps.repo_name" structure to record a dependency. 115 116- api 117 118The name of the API the dependency is specific to (e.g. "vulkan"). 119 120- url 121 122Specifies the URL of the repository. 123Example: https://github.com/KhronosGroup/Vulkan-Loader.git 124 125- sub_dir 126 127The directory where the program clones the repository, relative to 128the "top" directory. 129 130- build_dir 131 132The directory used to build the repository, relative to the "top" 133directory. 134 135- install_dir 136 137The directory used to store the installed build artifacts, relative 138to the "top" directory. 139 140- commit 141 142The commit used to checkout the repository. This can be a SHA-1 143object name or a refname used with the remote name "origin". 144 145- deps (optional) 146 147An array of pairs consisting of a CMake variable name and a 148repository name to specify a dependent repo and a "link" to 149that repo's install artifacts. For example: 150 151"deps" : [ 152 { 153 "var_name" : "VULKAN_HEADERS_INSTALL_DIR", 154 "repo_name" : "Vulkan-Headers" 155 } 156] 157 158which represents that this repository depends on the Vulkan-Headers 159repository and uses the VULKAN_HEADERS_INSTALL_DIR CMake variable to 160specify the location where it expects to find the Vulkan-Headers install 161directory. 162Note that the "repo_name" element must match the "name" element of some 163other repository in the JSON file. 164 165- prebuild (optional) 166- prebuild_linux (optional) (For Linux and MacOS) 167- prebuild_windows (optional) 168 169A list of commands to execute before building a dependent repository. 170This is useful for repositories that require the execution of some 171sort of "update" script or need to clone an auxillary repository like 172googletest. 173 174The commands listed in "prebuild" are executed first, and then the 175commands for the specific platform are executed. 176 177- custom_build (optional) 178 179A list of commands to execute as a custom build instead of using 180the built in CMake way of building. Requires "build_step" to be 181set to "custom" 182 183You can insert the following keywords into the commands listed in 184"custom_build" if they require runtime information (like whether the 185build config is "Debug" or "Release"). 186 187Keywords: 188{0} reference to a dictionary of repos and their attributes 189{1} reference to the command line arguments set before start 190{2} reference to the CONFIG_MAP value of config. 191 192Example: 193{2} returns the CONFIG_MAP value of config e.g. debug -> Debug 194{1}.config returns the config variable set when you ran update_dep.py 195{0}[Vulkan-Headers][repo_root] returns the repo_root variable from 196 the Vulkan-Headers GoodRepo object. 197 198- cmake_options (optional) 199 200A list of options to pass to CMake during the generation phase. 201 202- ci_only (optional) 203 204A list of environment variables where one must be set to "true" 205(case-insensitive) in order for this repo to be fetched and built. 206This list can be used to specify repos that should be built only in CI. 207 208- build_step (optional) 209 210Specifies if the dependent repository should be built or not. This can 211have a value of 'build', 'custom', or 'skip'. The dependent repositories are 212built by default. 213 214- build_platforms (optional) 215 216A list of platforms the repository will be built on. 217Legal options include: 218"windows" 219"linux" 220"darwin" 221"android" 222 223Builds on all platforms by default. 224 225Note 226---- 227 228The "sub_dir", "build_dir", and "install_dir" elements are all relative 229to the effective "top" directory. Specifying absolute paths is not 230supported. However, the "top" directory specified with the "--dir" 231option can be a relative or absolute path. 232 233""" 234 235import argparse 236import json 237import os 238import os.path 239import subprocess 240import sys 241import platform 242import multiprocessing 243import shlex 244import shutil 245import stat 246import time 247 248KNOWN_GOOD_FILE_NAME = 'known_good.json' 249 250CONFIG_MAP = { 251 'debug': 'Debug', 252 'release': 'Release', 253 'relwithdebinfo': 'RelWithDebInfo', 254 'minsizerel': 'MinSizeRel' 255} 256 257# NOTE: CMake also uses the VERBOSE environment variable. This is intentional. 258VERBOSE = os.getenv("VERBOSE") 259 260DEVNULL = open(os.devnull, 'wb') 261 262 263def on_rm_error( func, path, exc_info): 264 """Error handler for recursively removing a directory. The 265 shutil.rmtree function can fail on Windows due to read-only files. 266 This handler will change the permissions for the file and continue. 267 """ 268 os.chmod( path, stat.S_IWRITE ) 269 os.unlink( path ) 270 271def make_or_exist_dirs(path): 272 "Wrapper for os.makedirs that tolerates the directory already existing" 273 # Could use os.makedirs(path, exist_ok=True) if we drop python2 274 if not os.path.isdir(path): 275 os.makedirs(path) 276 277def command_output(cmd, directory): 278 # Runs a command in a directory and returns its standard output stream. 279 # Captures the standard error stream and prints it an error occurs. 280 # Raises a RuntimeError if the command fails to launch or otherwise fails. 281 if VERBOSE: 282 print('In {d}: {cmd}'.format(d=directory, cmd=cmd)) 283 284 result = subprocess.run(cmd, cwd=directory, capture_output=True, text=True) 285 286 if result.returncode != 0: 287 print(f'{result.stderr}', file=sys.stderr) 288 raise RuntimeError(f'Failed to run {cmd} in {directory}') 289 290 if VERBOSE: 291 print(result.stdout) 292 return result.stdout 293 294def run_cmake_command(cmake_cmd): 295 # NOTE: Because CMake is an exectuable that runs executables 296 # stdout/stderr are mixed together. So this combines the outputs 297 # and prints them properly in case there is a non-zero exit code. 298 result = subprocess.run(cmake_cmd, 299 stdout = subprocess.PIPE, 300 stderr = subprocess.STDOUT, 301 text = True 302 ) 303 304 if VERBOSE: 305 print(result.stdout) 306 print(f"CMake command: {cmake_cmd} ", flush=True) 307 308 if result.returncode != 0: 309 print(result.stdout, file=sys.stderr) 310 sys.exit(result.returncode) 311 312def escape(path): 313 return path.replace('\\', '/') 314 315class GoodRepo(object): 316 """Represents a repository at a known-good commit.""" 317 318 def __init__(self, json, args): 319 """Initializes this good repo object. 320 321 Args: 322 'json': A fully populated JSON object describing the repo. 323 'args': Results from ArgumentParser 324 """ 325 self._json = json 326 self._args = args 327 # Required JSON elements 328 self.name = json['name'] 329 self.url = json['url'] 330 self.sub_dir = json['sub_dir'] 331 self.commit = json['commit'] 332 # Optional JSON elements 333 self.build_dir = None 334 self.install_dir = None 335 if json.get('build_dir'): 336 self.build_dir = os.path.normpath(json['build_dir']) 337 if json.get('install_dir'): 338 self.install_dir = os.path.normpath(json['install_dir']) 339 self.deps = json['deps'] if ('deps' in json) else [] 340 self.prebuild = json['prebuild'] if ('prebuild' in json) else [] 341 self.prebuild_linux = json['prebuild_linux'] if ( 342 'prebuild_linux' in json) else [] 343 self.prebuild_windows = json['prebuild_windows'] if ( 344 'prebuild_windows' in json) else [] 345 self.custom_build = json['custom_build'] if ('custom_build' in json) else [] 346 self.cmake_options = json['cmake_options'] if ( 347 'cmake_options' in json) else [] 348 self.ci_only = json['ci_only'] if ('ci_only' in json) else [] 349 self.build_step = json['build_step'] if ('build_step' in json) else 'build' 350 self.build_platforms = json['build_platforms'] if ('build_platforms' in json) else [] 351 self.optional = set(json.get('optional', [])) 352 self.api = json['api'] if ('api' in json) else None 353 # Absolute paths for a repo's directories 354 dir_top = os.path.abspath(args.dir) 355 self.repo_dir = os.path.join(dir_top, self.sub_dir) 356 if self.build_dir: 357 self.build_dir = os.path.join(dir_top, self.build_dir) 358 if self.install_dir: 359 self.install_dir = os.path.join(dir_top, self.install_dir) 360 361 # By default the target platform is the host platform. 362 target_platform = platform.system().lower() 363 # However, we need to account for cross-compiling. 364 for cmake_var in self._args.cmake_var: 365 if "android.toolchain.cmake" in cmake_var: 366 target_platform = 'android' 367 368 self.on_build_platform = False 369 if self.build_platforms == [] or target_platform in self.build_platforms: 370 self.on_build_platform = True 371 372 def Clone(self, retries=10, retry_seconds=60): 373 if VERBOSE: 374 print('Cloning {n} into {d}'.format(n=self.name, d=self.repo_dir)) 375 for retry in range(retries): 376 make_or_exist_dirs(self.repo_dir) 377 try: 378 command_output(['git', 'clone', self.url, '.'], self.repo_dir) 379 # If we get here, we didn't raise an error 380 return 381 except RuntimeError as e: 382 print("Error cloning on iteration {}/{}: {}".format(retry + 1, retries, e)) 383 if retry + 1 < retries: 384 if retry_seconds > 0: 385 print("Waiting {} seconds before trying again".format(retry_seconds)) 386 time.sleep(retry_seconds) 387 if os.path.isdir(self.repo_dir): 388 print("Removing old tree {}".format(self.repo_dir)) 389 shutil.rmtree(self.repo_dir, onerror=on_rm_error) 390 continue 391 392 # If we get here, we've exhausted our retries. 393 print("Failed to clone {} on all retries.".format(self.url)) 394 raise e 395 396 def Fetch(self, retries=10, retry_seconds=60): 397 for retry in range(retries): 398 try: 399 command_output(['git', 'fetch', 'origin'], self.repo_dir) 400 # if we get here, we didn't raise an error, and we're done 401 return 402 except RuntimeError as e: 403 print("Error fetching on iteration {}/{}: {}".format(retry + 1, retries, e)) 404 if retry + 1 < retries: 405 if retry_seconds > 0: 406 print("Waiting {} seconds before trying again".format(retry_seconds)) 407 time.sleep(retry_seconds) 408 continue 409 410 # If we get here, we've exhausted our retries. 411 print("Failed to fetch {} on all retries.".format(self.url)) 412 raise e 413 414 def Checkout(self): 415 if VERBOSE: 416 print('Checking out {n} in {d}'.format(n=self.name, d=self.repo_dir)) 417 418 if self._args.do_clean_repo: 419 if os.path.isdir(self.repo_dir): 420 shutil.rmtree(self.repo_dir, onerror = on_rm_error) 421 if not os.path.exists(os.path.join(self.repo_dir, '.git')): 422 self.Clone() 423 self.Fetch() 424 if len(self._args.ref): 425 command_output(['git', 'checkout', self._args.ref], self.repo_dir) 426 else: 427 command_output(['git', 'checkout', self.commit], self.repo_dir) 428 429 if VERBOSE: 430 print(command_output(['git', 'status'], self.repo_dir)) 431 432 def CustomPreProcess(self, cmd_str, repo_dict): 433 return cmd_str.format(repo_dict, self._args, CONFIG_MAP[self._args.config]) 434 435 def PreBuild(self): 436 """Execute any prebuild steps from the repo root""" 437 for p in self.prebuild: 438 command_output(shlex.split(p), self.repo_dir) 439 if platform.system() == 'Linux' or platform.system() == 'Darwin': 440 for p in self.prebuild_linux: 441 command_output(shlex.split(p), self.repo_dir) 442 if platform.system() == 'Windows': 443 for p in self.prebuild_windows: 444 command_output(shlex.split(p), self.repo_dir) 445 446 def CustomBuild(self, repo_dict): 447 """Execute any custom_build steps from the repo root""" 448 449 # It's not uncommon for builds to not support universal binaries 450 if self._args.OSX_ARCHITECTURES: 451 print("Universal Binaries not supported for custom builds", file=sys.stderr) 452 exit(-1) 453 454 for p in self.custom_build: 455 cmd = self.CustomPreProcess(p, repo_dict) 456 command_output(shlex.split(cmd), self.repo_dir) 457 458 def CMakeConfig(self, repos): 459 """Build CMake command for the configuration phase and execute it""" 460 if self._args.do_clean_build: 461 if os.path.isdir(self.build_dir): 462 shutil.rmtree(self.build_dir, onerror=on_rm_error) 463 if self._args.do_clean_install: 464 if os.path.isdir(self.install_dir): 465 shutil.rmtree(self.install_dir, onerror=on_rm_error) 466 467 # Create and change to build directory 468 make_or_exist_dirs(self.build_dir) 469 os.chdir(self.build_dir) 470 471 cmake_cmd = [ 472 'cmake', self.repo_dir, 473 '-DCMAKE_INSTALL_PREFIX=' + self.install_dir 474 ] 475 476 # Allow users to pass in arbitrary cache variables 477 for cmake_var in self._args.cmake_var: 478 pieces = cmake_var.split('=', 1) 479 cmake_cmd.append('-D{}={}'.format(pieces[0], pieces[1])) 480 481 # For each repo this repo depends on, generate a CMake variable 482 # definitions for "...INSTALL_DIR" that points to that dependent 483 # repo's install dir. 484 for d in self.deps: 485 dep_commit = [r for r in repos if r.name == d['repo_name']] 486 if len(dep_commit) and dep_commit[0].on_build_platform: 487 cmake_cmd.append('-D{var_name}={install_dir}'.format( 488 var_name=d['var_name'], 489 install_dir=dep_commit[0].install_dir)) 490 491 # Add any CMake options 492 for option in self.cmake_options: 493 cmake_cmd.append(escape(option.format(**self.__dict__))) 494 495 # Set build config for single-configuration generators (this is a no-op on multi-config generators) 496 cmake_cmd.append(f'-D CMAKE_BUILD_TYPE={CONFIG_MAP[self._args.config]}') 497 498 if self._args.OSX_ARCHITECTURES: 499 # CMAKE_OSX_ARCHITECTURES must be a semi-colon seperated list 500 cmake_osx_archs = self._args.OSX_ARCHITECTURES.replace(':', ';') 501 cmake_cmd.append(f'-D CMAKE_OSX_ARCHITECTURES={cmake_osx_archs}') 502 503 # Use the CMake -A option to select the platform architecture 504 # without needing a Visual Studio generator. 505 if platform.system() == 'Windows' and self._args.generator != "Ninja": 506 if self._args.arch.lower() == '64' or self._args.arch == 'x64' or self._args.arch == 'win64': 507 cmake_cmd.append('-A') 508 cmake_cmd.append('x64') 509 else: 510 cmake_cmd.append('-A') 511 cmake_cmd.append('Win32') 512 513 # Apply a generator, if one is specified. This can be used to supply 514 # a specific generator for the dependent repositories to match 515 # that of the main repository. 516 if self._args.generator is not None: 517 cmake_cmd.extend(['-G', self._args.generator]) 518 519 # Removes warnings related to unused CLI 520 # EX: Setting CMAKE_CXX_COMPILER for a C project 521 if not VERBOSE: 522 cmake_cmd.append("--no-warn-unused-cli") 523 524 run_cmake_command(cmake_cmd) 525 526 def CMakeBuild(self): 527 """Build CMake command for the build phase and execute it""" 528 cmake_cmd = ['cmake', '--build', self.build_dir, '--target', 'install', '--config', CONFIG_MAP[self._args.config]] 529 if self._args.do_clean: 530 cmake_cmd.append('--clean-first') 531 532 # Xcode / Ninja are parallel by default. 533 if self._args.generator != "Ninja" or self._args.generator != "Xcode": 534 cmake_cmd.append('--parallel') 535 cmake_cmd.append(format(multiprocessing.cpu_count())) 536 537 run_cmake_command(cmake_cmd) 538 539 def Build(self, repos, repo_dict): 540 """Build the dependent repo and time how long it took""" 541 if VERBOSE: 542 print('Building {n} in {d}'.format(n=self.name, d=self.repo_dir)) 543 print('Build dir = {b}'.format(b=self.build_dir)) 544 print('Install dir = {i}\n'.format(i=self.install_dir)) 545 546 start = time.time() 547 548 self.PreBuild() 549 550 if self.build_step == 'custom': 551 self.CustomBuild(repo_dict) 552 else: 553 self.CMakeConfig(repos) 554 self.CMakeBuild() 555 556 total_time = time.time() - start 557 558 print(f"Installed {self.name} ({self.commit}) in {total_time} seconds", flush=True) 559 560 def IsOptional(self, opts): 561 return len(self.optional.intersection(opts)) > 0 562 563def GetGoodRepos(args): 564 """Returns the latest list of GoodRepo objects. 565 566 The known-good file is expected to be in the same 567 directory as this script unless overridden by the 'known_good_dir' 568 parameter. 569 """ 570 if args.known_good_dir: 571 known_good_file = os.path.join( os.path.abspath(args.known_good_dir), 572 KNOWN_GOOD_FILE_NAME) 573 else: 574 known_good_file = os.path.join( 575 os.path.dirname(os.path.abspath(__file__)), KNOWN_GOOD_FILE_NAME) 576 with open(known_good_file) as known_good: 577 return [ 578 GoodRepo(repo, args) 579 for repo in json.loads(known_good.read())['repos'] 580 ] 581 582 583def GetInstallNames(args): 584 """Returns the install names list. 585 586 The known-good file is expected to be in the same 587 directory as this script unless overridden by the 'known_good_dir' 588 parameter. 589 """ 590 if args.known_good_dir: 591 known_good_file = os.path.join(os.path.abspath(args.known_good_dir), 592 KNOWN_GOOD_FILE_NAME) 593 else: 594 known_good_file = os.path.join( 595 os.path.dirname(os.path.abspath(__file__)), KNOWN_GOOD_FILE_NAME) 596 with open(known_good_file) as known_good: 597 install_info = json.loads(known_good.read()) 598 if install_info.get('install_names'): 599 return install_info['install_names'] 600 else: 601 return None 602 603 604def CreateHelper(args, repos, filename): 605 """Create a CMake config helper file. 606 607 The helper file is intended to be used with 'cmake -C <file>' 608 to build this home repo using the dependencies built by this script. 609 610 The install_names dictionary represents the CMake variables used by the 611 home repo to locate the install dirs of the dependent repos. 612 This information is baked into the CMake files of the home repo and so 613 this dictionary is kept with the repo via the json file. 614 """ 615 install_names = GetInstallNames(args) 616 with open(filename, 'w') as helper_file: 617 for repo in repos: 618 # If the repo has an API tag and that does not match 619 # the target API then skip it 620 if repo.api is not None and repo.api != args.api: 621 continue 622 if install_names and repo.name in install_names and repo.on_build_platform: 623 helper_file.write('set({var} "{dir}" CACHE STRING "" FORCE)\n' 624 .format( 625 var=install_names[repo.name], 626 dir=escape(repo.install_dir))) 627 628 629def main(): 630 parser = argparse.ArgumentParser( 631 description='Get and build dependent repos at known-good commits') 632 parser.add_argument( 633 '--known_good_dir', 634 dest='known_good_dir', 635 help="Specify directory for known_good.json file.") 636 parser.add_argument( 637 '--dir', 638 dest='dir', 639 default='.', 640 help="Set target directory for repository roots. Default is \'.\'.") 641 parser.add_argument( 642 '--ref', 643 dest='ref', 644 default='', 645 help="Override 'commit' with git reference. E.g., 'origin/main'") 646 parser.add_argument( 647 '--no-build', 648 dest='do_build', 649 action='store_false', 650 help= 651 "Clone/update repositories and generate build files without performing compilation", 652 default=True) 653 parser.add_argument( 654 '--clean', 655 dest='do_clean', 656 action='store_true', 657 help="Clean files generated by compiler and linker before building", 658 default=False) 659 parser.add_argument( 660 '--clean-repo', 661 dest='do_clean_repo', 662 action='store_true', 663 help="Delete repository directory before building", 664 default=False) 665 parser.add_argument( 666 '--clean-build', 667 dest='do_clean_build', 668 action='store_true', 669 help="Delete build directory before building", 670 default=False) 671 parser.add_argument( 672 '--clean-install', 673 dest='do_clean_install', 674 action='store_true', 675 help="Delete install directory before building", 676 default=False) 677 parser.add_argument( 678 '--skip-existing-install', 679 dest='skip_existing_install', 680 action='store_true', 681 help="Skip build if install directory exists", 682 default=False) 683 parser.add_argument( 684 '--arch', 685 dest='arch', 686 choices=['32', '64', 'x86', 'x64', 'win32', 'win64'], 687 type=str.lower, 688 help="Set build files architecture (Visual Studio Generator Only)", 689 default='64') 690 parser.add_argument( 691 '--config', 692 dest='config', 693 choices=['debug', 'release', 'relwithdebinfo', 'minsizerel'], 694 type=str.lower, 695 help="Set build files configuration", 696 default='debug') 697 parser.add_argument( 698 '--api', 699 dest='api', 700 default='vulkan', 701 choices=['vulkan'], 702 help="Target API") 703 parser.add_argument( 704 '--generator', 705 dest='generator', 706 help="Set the CMake generator", 707 default=None) 708 parser.add_argument( 709 '--optional', 710 dest='optional', 711 type=lambda a: set(a.lower().split(',')), 712 help="Comma-separated list of 'optional' resources that may be skipped. Only 'tests' is currently supported as 'optional'", 713 default=set()) 714 parser.add_argument( 715 '--cmake_var', 716 dest='cmake_var', 717 action='append', 718 metavar='VAR[=VALUE]', 719 help="Add CMake command line option -D'VAR'='VALUE' to the CMake generation command line; may be used multiple times", 720 default=[]) 721 parser.add_argument( 722 '--osx-archs', 723 dest='OSX_ARCHITECTURES', 724 help="Architectures when building a universal binary. Takes a colon seperated list. Ex: arm64:x86_64", 725 type=str, 726 default=None) 727 728 args = parser.parse_args() 729 save_cwd = os.getcwd() 730 731 if args.OSX_ARCHITECTURES: 732 print(f"Building dependencies as universal binaries targeting {args.OSX_ARCHITECTURES}") 733 734 # Create working "top" directory if needed 735 make_or_exist_dirs(args.dir) 736 abs_top_dir = os.path.abspath(args.dir) 737 738 repos = GetGoodRepos(args) 739 repo_dict = {} 740 741 print('Starting builds in {d}'.format(d=abs_top_dir)) 742 for repo in repos: 743 # If the repo has an API tag and that does not match 744 # the target API then skip it 745 if repo.api is not None and repo.api != args.api: 746 continue 747 748 # If the repo has a platform whitelist, skip the repo 749 # unless we are building on a whitelisted platform. 750 if not repo.on_build_platform: 751 continue 752 753 # Skip building the repo if its install directory already exists 754 # and requested via an option. This is useful for cases where the 755 # install directory is restored from a cache that is known to be up 756 # to date. 757 if args.skip_existing_install and os.path.isdir(repo.install_dir): 758 print('Skipping build for repo {n} due to existing install directory'.format(n=repo.name)) 759 continue 760 761 # Skip test-only repos if the --tests option was not passed in 762 if repo.IsOptional(args.optional): 763 continue 764 765 field_list = ('url', 766 'sub_dir', 767 'commit', 768 'build_dir', 769 'install_dir', 770 'deps', 771 'prebuild', 772 'prebuild_linux', 773 'prebuild_windows', 774 'custom_build', 775 'cmake_options', 776 'ci_only', 777 'build_step', 778 'build_platforms', 779 'repo_dir', 780 'on_build_platform') 781 repo_dict[repo.name] = {field: getattr(repo, field) for field in field_list} 782 783 # If the repo has a CI whitelist, skip the repo unless 784 # one of the CI's environment variable is set to true. 785 if len(repo.ci_only): 786 do_build = False 787 for env in repo.ci_only: 788 if env not in os.environ: 789 continue 790 if os.environ[env].lower() == 'true': 791 do_build = True 792 break 793 if not do_build: 794 continue 795 796 # Clone/update the repository 797 repo.Checkout() 798 799 # Build the repository 800 if args.do_build and repo.build_step != 'skip': 801 repo.Build(repos, repo_dict) 802 803 # Need to restore original cwd in order for CreateHelper to find json file 804 os.chdir(save_cwd) 805 CreateHelper(args, repos, os.path.join(abs_top_dir, 'helper.cmake')) 806 807 sys.exit(0) 808 809 810if __name__ == '__main__': 811 main() 812 813