Skip to content
Snippets Groups Projects
  1. Jan 03, 2024
  2. Jan 16, 2023
  3. Nov 07, 2012
    • Lawrence Crowl's avatar
      Add a contrib script for comparing the performance of two sets of · bff0e529
      Lawrence Crowl authored
      compiler runs.
      
      Usage documentation is in the script.
      
      The script produces output of the form:
      
      $ compare_two_ftime_report_sets "Log0/*perf" "Log3/*perf" 
      
      Arithmetic sample for timevar log files
      "Log0/*perf"
      and selecting lines containing "TOTAL" with desired confidence 95 is 
      trial count is 4, mean is 443.022 (95% confidence in 440.234 to 445.811),
      std.deviation is 1.75264, std.error is 0.876322
      
      Arithmetic sample for timevar log files
      "Log3/*perf"
      and selecting lines containing "TOTAL" with desired confidence 95 is 
      trial count is 4, mean is 441.302 (95% confidence in 436.671 to 445.934),
      std.deviation is 2.91098, std.error is 1.45549
      
      The first sample appears to be 0.39% larger,
      with 60% confidence of being larger.
      To reach 95% confidence, you need roughly 14 trials,
      assuming the standard deviation is stable, which is iffy.
      
      Tested on x86_64 builds.
      
      
      Index: contrib/ChangeLog
      
      2012-11-05  Lawrence Crowl  <crowl@google.com>
      
      	* compare_two_ftime_report_sets: New.
      
      From-SVN: r193277
      bff0e529
Loading