Environment Intel Fortran 11.1
1. How to create a FORTRAN DLL in visual studio with INTEL FORTRAN compiler:
http://sukhbinder.wordpress.com/2011/04/14/how-to-create-fortran-dll-in-visual-studio-with-intel-fortran-compiler/
Simple to follow, worked.
2. How to call FORTRAN DLL in visual studio with INTEL FORTRAN compiler:
http://www.eng-tips.com/viewthread.cfm?qid=117348
Another method:
http://www.tek-tips.com/viewthread.cfm?qid=1572697
Calling DLL from C#
http://software.intel.com/en-us/articles/calling-fortran-function-or-subroutine-in-dll-from-c-code/
Thursday, April 19, 2012
Parallel Programming 101 - Fortran OpenMP for dummies
These content are obtained from the internet, I only extracted the most useful information and tested them. I do not claim any authorship.
1. Compiler Settings
Compaq Visual Fortran compiler does not support parallel programming feature.
If you are using Intel Visual Fortran to compile your parallel code, you have to do the following in order to have it run correctly:
Right click on project
2. To convert a serial code into a parallel code, you will need to use OpenMP directives
Basic format of OpenMP directives:
A typical OpenMP directive looks like this (Fortran is not case-sensitive):
!$OMP PARALLEL DO
OpenMP directives must start with a sentinel !$OMP
!$OMP can appear in any column, but must be preceded by white space only
Continuation lines must have an ampersand as the last non-blank character in a line.
!$OMP PARALLEL PRIVATE(TID)can be written as
!$OMP PARALLEL &
!$OMP PRIVATE (TID)
(Although directives start with an ! sign, it is not a comment, as in Fortran)
3. Parallel Region Construct
Parallel region: a block of code that will be executed by multiple threads.
When a thread reaches a parallel region, it creates a team of threads. The code within the parallel region is duplicated and all threads will execute the code.
The format of the parallel construct is
Example Code 1
PROGRAM Hello
!$OMP PARALLEL
print*, 'Hello World!'
!$OMP END PARALLEL
END PROGRAM
If the above example is running on a duo core computer, then
the output will be:
Hello World!
Hello World!
(but this is not useful at all so far, as we do not want to repeat the same task on all threads)
!$omp parallel
TID = OMP_GET_THREAD_NUM()
PRINT *, 'Hello World from thread = ', TID
!$omp end parallel
4. Work-Sharing Construct
A work-sharing construct does not create new threads. It divides the execution of the enclosed code among the existing threads.
Three types of work-sharing construct
(Now it is getting interesting!)
4.1 SINGLE Directive
The SINGLE directive speci…es that the enclosed code is to be executed by only one thread in the team.
Useful when dealing with sections of code that are not thread safe (such as I/O)
Example
!$OMP PARALLEL
!$OMP SINGLE
!$OMP END SINGLE
!$OMP END PARALLEL
4.2 DO directives
The DO directive specifi…es that the iterations of the loop immediately following it must be executed in parallel by the team.
The format of the DO directive is
!$OMP DO
[Code Block]
!$OMP END DO
Example Code
!$OMP PARALLEL
!$OMP DO
DO i = 1,n
z(i) = a*x(i) + y
END DO
!$OMP END DO
!$OMP END PARALLEL
For convenience, there is
PARALLEL DO
We can replace the above code with
!$OMP PARALLEL DO
DO i = 1,n
z(i) = a*x(i) + y
END DO
Note that PARALLEL DO does not need an END PARALLEL DO.
4.3 SECTION directives
The SECTIONS directive divides the enclosed sections of code among the existing threads.
The format of the SECTIONS directive is
!$OMP SECTIONS
!$OMP SECTION
[Code Block 1]
!$OMP SECTION
[Code Block 2]
!$OMP SECTION
[Code Block 3]
!$OMP END SECTIONS
Example Code
!$OMP PARALLEL SHARED(A,B,C,D), PRIVATE(I)
!$OMP SECTIONS
!$OMP SECTION
DO I = 1, N
C(I) = A(I) + B(I)
ENDDO
!$OMP SECTION
DO I = 1, N
D(I) = A(I) * B(I)
ENDDO
!$OMP END SECTIONS
!$OMP END PARALLEL
For convenience, there is
PARALLEL SECTIONS
We can replace the above code with
!$OMP PARALLEL SECTIONS
!$OMP SECTION
[Code Block 1]
!$OMP SECTION
[Code Block 2]
!$OMP END PARALLEL SECTIONS
With the above work-share construct, I can construct some simple yet useful program.
----------------------------------------------------------------------------
Trouble 1:
When I compile the program and sent my executable to a lab computer with 8 cores to run, it failed with an error message:
This is the most useful answer I found on the internet
Even when static libraries are selected, the OpenMP library is linked dynamically - this can be overridden by adding /Qopenmp-link:static under Fortran > Command Line > Additional Options.
Directives for Sychronization
CRITICAL contains the codes that will be executed in serial, no longer parallel, this is to avoid data comfliction
!$OMP CRITICAL
...
!$OMP END CRITICAL
Example (I did not test this, probably should)
! Look for the largest element in an array
currentMax = -1.E10
!$OMP PARALLEL DO
DO i = 1, n
IF( a(i) < currentMax) THEN
!$OMP CRITICAL
IF( a(i) < currentMax) THEN
currentMax = a(i)
ENDIF
!$OMP END CRITICAL
END IF
END DO
!$OMP END PARALLEL DO
--------------------------
I am running out of time for another project. This will be put on hold.
References:
http://www.amazon.com/Parallel-Programming-OpenMP-Rohit-Chandra/dp/1558606718#reader_1558606718
http://openmp.org/mp-documents/omp-hands-on-SC08.pdf
http://www.cita.utoronto.ca/MISC/computing_guide/docs/Intel_Fortran_v8.1/f_ug2/index.htm#par_dirs.htm
http://www.cita.utoronto.ca/MISC/computing_guide/docs/Intel_Fortran_v8.1/f_ug2/par_dirs.htm
https://computing.llnl.gov/tutorials/openMP/#RunTimeLibrary
https://computing.llnl.gov/tutorials/openMP/exercise.html
https://computing.llnl.gov/?set=training&page=index#training_materials
https://computing.llnl.gov/tutorials/openMP/exercise.html
1. Compiler Settings
Compaq Visual Fortran compiler does not support parallel programming feature.
If you are using Intel Visual Fortran to compile your parallel code, you have to do the following in order to have it run correctly:
Right click on project
Choose properties -> Fortran -> Preprocesor -> OpenMP->Conditional Compilation -> Yes
Choose properties -> Fortran -> Language -> Process->OpenMP Directives -> Generate Parallell Code(/Qopenmp)
(- I have to do these in my intel fortran compiler to compile! Also it is best to use free format Fortran)
2. To convert a serial code into a parallel code, you will need to use OpenMP directives
Basic format of OpenMP directives:
A typical OpenMP directive looks like this (Fortran is not case-sensitive):
!$OMP PARALLEL DO
OpenMP directives must start with a sentinel !$OMP
!$OMP can appear in any column, but must be preceded by white space only
Continuation lines must have an ampersand as the last non-blank character in a line.
!$OMP PARALLEL PRIVATE(TID)can be written as
!$OMP PARALLEL &
!$OMP PRIVATE (TID)
(Although directives start with an ! sign, it is not a comment, as in Fortran)
3. Parallel Region Construct
Parallel region: a block of code that will be executed by multiple threads.
When a thread reaches a parallel region, it creates a team of threads. The code within the parallel region is duplicated and all threads will execute the code.
The format of the parallel construct is
!$OMP PARALLEL
[Code Block]
!$OMP END PARALLEL
[Code Block]
!$OMP END PARALLEL
Example Code 1
PROGRAM Hello
!$OMP PARALLEL
print*, 'Hello World!'
END PROGRAM
If the above example is running on a duo core computer, then
the output will be:
Hello World!
Hello World!
(but this is not useful at all so far, as we do not want to repeat the same task on all threads)
Here
is my first OpenMP program, it runs, but when I wanted to loop through
the threads, error occurred. So I went to study a bit more. Note that
you need to include omp_lib.h in this program. This has not touched so far.
!----------------------------
program ompHelloWorld
#ifdef _OPENMP
include 'omp_lib.h' !needed for OMP_GET_NUM_THREADS()
#endif
integer TID
#ifdef _OPENMP
include 'omp_lib.h' !needed for OMP_GET_NUM_THREADS()
#endif
integer TID
!$omp parallel
TID = OMP_GET_THREAD_NUM()
PRINT *, 'Hello World from thread = ', TID
!$omp end parallel
continue
end program ompHelloWorld
end program ompHelloWorld
!--------------------
Increasing the complexity
!--------------------
program ompHelloWorld
#ifdef _OPENMP
include 'omp_lib.h' !needed for OMP_GET_NUM_THREADS()
#endif
integer nthr
integer NTHREADS, TID
TID=-999
NTHREADS=-999
!$omp parallel
TID = OMP_GET_THREAD_NUM()
PRINT *, 'Hello World from thread = ', TID
IF (TID .EQ. 0) THEN
NTHREADS = OMP_GET_NUM_THREADS()
PRINT *, 'Master computer report:'
PRINT *, 'Number of threads = ', NTHREADS
ELSE
PRINT *, 'Slave computer report:'
PRINT *, 'I am on vacation.'
END IF
!$omp end parallel
continue
end program ompHelloWorld
#ifdef _OPENMP
include 'omp_lib.h' !needed for OMP_GET_NUM_THREADS()
#endif
integer nthr
integer NTHREADS, TID
TID=-999
NTHREADS=-999
!$omp parallel
TID = OMP_GET_THREAD_NUM()
PRINT *, 'Hello World from thread = ', TID
IF (TID .EQ. 0) THEN
NTHREADS = OMP_GET_NUM_THREADS()
PRINT *, 'Master computer report:'
PRINT *, 'Number of threads = ', NTHREADS
ELSE
PRINT *, 'Slave computer report:'
PRINT *, 'I am on vacation.'
END IF
!$omp end parallel
continue
end program ompHelloWorld
!--------------------
(Note that there is an issue with using F10 to step through the code or using F5 to debug the code. It appears that F10 stepping results are not correct. Also, inserting breakpoints into the code seems to cause issue too. I have to visit this issue!!!)
4. Work-Sharing Construct
A work-sharing construct does not create new threads. It divides the execution of the enclosed code among the existing threads.
Three types of work-sharing construct
- DO directive:
- SECTIONS directive
- SINGLE directive
(Now it is getting interesting!)
4.1 SINGLE Directive
The SINGLE directive speci…es that the enclosed code is to be executed by only one thread in the team.
Useful when dealing with sections of code that are not thread safe (such as I/O)
Example
!$OMP PARALLEL
!$OMP SINGLE
!$OMP END SINGLE
!$OMP END PARALLEL
4.2 DO directives
The DO directive specifi…es that the iterations of the loop immediately following it must be executed in parallel by the team.
The format of the DO directive is
!$OMP DO
[Code Block]
!$OMP END DO
Example Code
!$OMP PARALLEL
!$OMP DO
DO i = 1,n
z(i) = a*x(i) + y
END DO
!$OMP END DO
!$OMP END PARALLEL
For convenience, there is
PARALLEL DO
We can replace the above code with
!$OMP PARALLEL DO
DO i = 1,n
z(i) = a*x(i) + y
END DO
Note that PARALLEL DO does not need an END PARALLEL DO.
4.3 SECTION directives
The SECTIONS directive divides the enclosed sections of code among the existing threads.
The format of the SECTIONS directive is
!$OMP SECTIONS
!$OMP SECTION
[Code Block 1]
!$OMP SECTION
[Code Block 2]
!$OMP SECTION
[Code Block 3]
!$OMP END SECTIONS
Example Code
!$OMP PARALLEL SHARED(A,B,C,D), PRIVATE(I)
!$OMP SECTIONS
!$OMP SECTION
DO I = 1, N
C(I) = A(I) + B(I)
ENDDO
!$OMP SECTION
DO I = 1, N
D(I) = A(I) * B(I)
ENDDO
!$OMP END SECTIONS
!$OMP END PARALLEL
For convenience, there is
PARALLEL SECTIONS
We can replace the above code with
!$OMP PARALLEL SECTIONS
!$OMP SECTION
[Code Block 1]
!$OMP SECTION
[Code Block 2]
!$OMP END PARALLEL SECTIONS
With the above work-share construct, I can construct some simple yet useful program.
----------------------------------------------------------------------------
Trouble 1:
When I compile the program and sent my executable to a lab computer with 8 cores to run, it failed with an error message:
libiomp5md.dll is missing
This is the most useful answer I found on the internet
Even when static libraries are selected, the OpenMP library is linked dynamically - this can be overridden by adding /Qopenmp-link:static under Fortran > Command Line > Additional Options.
OMP libary
!$omp parallel num_threads(3) !use three threads for this programDirectives for Sychronization
CRITICAL contains the codes that will be executed in serial, no longer parallel, this is to avoid data comfliction
!$OMP CRITICAL
...
!$OMP END CRITICAL
Example (I did not test this, probably should)
! Look for the largest element in an array
currentMax = -1.E10
!$OMP PARALLEL DO
DO i = 1, n
IF( a(i) < currentMax) THEN
!$OMP CRITICAL
IF( a(i) < currentMax) THEN
currentMax = a(i)
ENDIF
!$OMP END CRITICAL
END IF
END DO
!$OMP END PARALLEL DO
--------------------------
I am running out of time for another project. This will be put on hold.
References:
- A comprehensive Book:
http://www.amazon.com/Parallel-Programming-OpenMP-Rohit-Chandra/dp/1558606718#reader_1558606718
- A simple to follow yet in-depth ppt (in C)
http://openmp.org/mp-documents/omp-hands-on-SC08.pdf
- Useful table summaries of directives
http://www.cita.utoronto.ca/MISC/computing_guide/docs/Intel_Fortran_v8.1/f_ug2/index.htm#par_dirs.htm
http://www.cita.utoronto.ca/MISC/computing_guide/docs/Intel_Fortran_v8.1/f_ug2/par_dirs.htm
- OpenMP training workshop of Llnl
https://computing.llnl.gov/tutorials/openMP/#RunTimeLibrary
https://computing.llnl.gov/tutorials/openMP/exercise.html
https://computing.llnl.gov/?set=training&page=index#training_materials
https://computing.llnl.gov/tutorials/openMP/exercise.html
Tuesday, April 17, 2012
Subscribe to:
Posts (Atom)