1. Sept 12 .Describe two articles in statistics literature that report Monte Carlo studies
paper 1:
Title: Simulation Budget Allocation for Further Enhancing the Efficiency of Ordinal Optimization
Author: Chun-Hung Chen, J. Lin, E. Yucesan, and S. E. Chick
Journal of Discrete Event Dynamic Systems: Theory and Applications, V. 10, #3, pp 251-270, July 2000
Abstract—Ordinal Optimization has emerged as an efficient technique for simulation and optimization. Exponential convergence rates can be achieved in many cases. In this paper, we present a new approach that can further enhance the efficiency of ordinal optimization. Our approach determines a highly efficient number of simulation replications or samples and significantly reduces the total simulation cost. We also compare several different allocation procedures, including a popular two-stage procedure in simulation literature. Numerical testing shows that our approach is much more efficient than all compared methods. The results further indicate that our approach can obtain a speedup factor of higher than 20 above and beyond the speedup achieved by the use of ordinal optimization for a 210-design example.
paper 2:
Title: Computing Efforts Allocation for Ordinal Optimization and Discrete Event Simulation
Author: Hsiao-Chang Chen, Chun-Hung Chen, and Enver Yucesan
IEEE Transactions on Automatic Control, Vol. 45, No. 5, May 2000
Abstract—Ordinal optimization has emerged as an efficient technique for simulation and optimization. Exponential convergence rates can be achieved in many cases. In this paper, we present a new approach that
can further enhance the efficiency of ordinal optimization. Our approach intelligently determines the optimal number of simulation replications (or samples) and significantly reduces the total simulation cost. Numerical
illustrations are included. The results indicate that our approach can obtain an additional 74% computation time reduction above and beyond the reduction obtained through the use of ordinal optimization for a 10-design example.
2. Sept 19. Design a plan to replicate and extend one of the studies
1. Read and understand the OCBA method.
2. replicate OCBA method and other four traditional methods.
3. extend the method by changing the definition of P(CS). We change the definition of P(CS) to the probability of correctly selecting all of the top-M designs, i.e., we want to select both the best, the second best, until Mth best designs, not just the best design as in the paper. I will try to find a solution like something given in Theorem 1 and use Monte Carlo study to test the robustness of OCBA method.
3. Sept 26. Feasibility study -- software, etc.
I use C for coding because C is fast (which is ideal for Monta Carlo study) and I am familiar with it. Although C doesn't have powerful random number generation functions, but the user group is big, and internet search shows that random number generation functions for almost all the needed distribution have already been developed.
4. Oct 10. Conduct study.
Replication: I finished the comparison of OCBA method and equal allocation in numerical experiment 1. The result matches the one given by paper.
| T | P_OCBA | P_equal |
| 200 | 0.753 | 0.628 |
| 400 | 0.889 | 0.709 |
| 600 | 0.948 | 0.806 |
| 800 | 0.974 | 0.863 |
| 1000 | 0.985 | 0.881 |
| 1200 | 0.993 | 0.903 |
| 1400 | 0.995 | 0.937 |
| 1600 | 0.997 | 0.934 |
| 1800 | 0.997 | 0.938 |
| 2000 | 1 | 0.956 |
5. Nov 14
6. Nov 28