Code Monkey home page Code Monkey logo

mustem's Introduction

μSTEM

μSTEM is a transmission electron microscopy (TEM) simulation suite, in particular for scanning transmission electron microscopy (STEM) images, that was developed mainly at the University of Melbourne. The computing suite is based on the multislice method. More detail can be found in the manual and the following scientific paper:

Modelling the inelastic scattering of fast electrons, L.J. Allen, A.J. D'Alfonso and S.D. Findlay, Ultramicroscopy, Vol. 151, pp. 11-22, (2015).

PACBED pattern

Prerequisites

GPU version

To run precompiled executables

  • 64 bit windows OS
  • A CUDA enabled GPU with compute class 3.0 or greater
  • Version 9.0 of the Nvidia CUDA toolkit, this installs the .dll files necessary to run the GPU versions of μSTEM. Future versions of μSTEM will likely use later versions (9.1 and above) of the CUDA toolkit so this is a site you might have to revisit in future.

To compile source code

  • PGI Visual Fortran
  • A CUDA enabled GPU
  • Any operating system that the PGI compiler supports, this includes Windows as well as most versions of Linux and MacOS

CPU version

  • 64 bit Windows OS

To compile source code

  • Any Fortran 90 compiler and FFTW libraries
  • Any operating system

Precompiled executables

Precompiled executables can be found in the Executables folder of the repository or by clicking the links below:

Tutorial

Click here to download the μSTEM tutorial calculations, powerpoint and activity sheet.

Compiling source code

MuSTEM is built using the PGI Fortran compiler and Microsoft Visual Studio 2015, please make sure that this software is correctly installed before proceeding. Create a new Visual Studio project and add the source code contained in this repository. The GPU version of the code requires the source files in the GPU_routines folder and the CPU only version of the code requires the source files in the CPU_routines folder. Modify the project properties so that Microsoft Visual Studio passes the following commands to the PGI compiler:

Build commands:

-Mpreprocess -Bstatic -Mbackslash -mp -Mcuda=cuda9.0 -I"C:\Program Files\PGI\win64\17.3\include" -I"c:\program files\pgi\win64\17.3\include" -I"C:\Program Files\PGI\Microsoft Open Tools 14\include" -I"C:\Program Files (x86)\Windows Kits\10\Include\shared" -I"C:\Program Files (x86)\Windows Kits\10\Include\um" -fast -ta=tesla -Minform=warn

To build the single precision version add the command -Dsingle_precision. For the Double precision version add the command -Ddouble_precision. To build the GPU version (requires the PGI compiler) add the command -Dgpu. The code also requires recursive routines to be enabled (The /recursive command in the Intel Visual Fortran compiler) for correct calculation of the absorptive form factors.

Linker commands:

-Bstatic -mp -Mcuda=cuda8.0 -ta=tesla -o"$(Outdir)\MuSTEM_Open.exe" -Wl,/libpath:"C:\Program Files\PGI\win64\17.3\lib" -Wl,/libpath:"C:\Program Files\PGI\win64\2017\cuda\9.0\lib64" cufft.lib

The links to the PGI CUDA libraries and Windows kits may need to be modified depending on the install directories. $(Outdir) represents the output directory of the Microsoft Visual Studio build.

Some example simulations are included on the MuSTEM website.

Contributing

Please contact Dr. Hamish Brown with project suggestions and contributions.

Authors

  • Professor Les J Allen
  • Dr. Hamish G Brown - Maintainer
  • Dr. Adrian J D'Alfonso
  • Dr. Scott D Findlay
  • Dr. Ben D Forbes

License

This project is licensed under the GNU GPLv3.0 - see the LICENSE.txt file for details.

Acknowledgments

Les Allen, Adrian D'Alfonso and Scott Findlay originally took the initiative to make this code publicly available, with Adrian D'Alfonso taking on the considerable task of setting up the GPU code. Hamish Brown and Ben Forbes have subsequently made substantial refinements and extensions, with Ben Forbes responsible for several efficiency improvements. The code was developed mainly at the University of Melbourne. We would like to acknowledge the contributions of Mark Oxley and Chris Rossouw to the theoretical and numerical aspects of collaborative research that underpins μSTEM

In particular, the code IONIZER, whose primary author has been Mark Oxley, was used to set up the parametrized atomic scattering factors for ionization used in μSTEM. Eireann Cosgriff, Torgny Josefsson, Nathan Lugg, Andrew Martin, Gary Ruben and Chris Witte have been members of the group at Melbourne University at various stages and all made contributions to our understanding of inelastic scattering and the simulation thereof.

mustem's People

Contributors

hamishgbrown avatar xjtuwj avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

mustem's Issues

a setting question about PACBED pattern

Hi Hamish, I have a naive question about PACBED pattern. how could I set numbers of scan positions when simulating, I only can know the scan positions from the output files. Thanks a lot.

Bug of GPU version

In windows GPU version, when running in 'Record' mode, the program will crash after entering accelerating voltage. The error message is as follows:
FIO/stdio: No such file or directory FIO-F-/formatted write/unit=6/error code returned by host stdio - 2. File name = stdout formatted, sequential access record = 107 In source file C:\Users\hbro0001\2017_4_4_MuSTEM_v_4_8\v_4_9_for_GIT\licensed\Source\mod_global_variables.f90, at line number 146
image

SIGSEGV, segmentation fault occurred

Hello,

I compiled the software on CentoOS7 machine via:

cd ~/MuSTEM/Source
module load FFTW/3.3.10-gompi-2021b
module load ifort/2019.1.144-GCC-8.2.0-2.31.1
module load imkl/2022.0.1
mv Makefile.linux Makefile
make all

Specs of the compute node where the software is being run is this:
compute-3-0.extreme

 state = job-exclusive

 power_state = Running

 np = 16

 properties = compute-g1,mie_shared_g1

 ntype = cluster

 jobs = 0-15/971892.admin.extreme.acer.uic.edu

 status = opsys=linux,uname=Linux compute-3-0.extreme 3.10.0-1160.49.1.el7.x86_64 #1 SMP Tue Nov 30 15:51:32 UTC 2021 x86_64,sessions=2949 5262 5641 19237 27134 30005 30499,nsessions=7,nusers=2,idletime=1119833,totmem=135974088kb,availmem=128468960kb,physmem=131779788kb,ncpus=16,loadave=16.01,gres=,netload=2078640449540,state=free,varattr= ,cpuclock=Fixed,version=6.1.3,rectime=1660059045,jobs=971892.admin.extreme.acer.uic.edu

vi HRTEM_example.txt

Output filename
Results\HRTEM_example
Input crystal file name
SrTiO3_001.xtl
Probe accelerating voltage (kV)
300
Slice unit cell <1> yes <2> no
1
Number of distinct potentials
2
<1> manual or <2> auto slicing
1
depths
0.000000
depths
0.5000000
Thickness
50:200:50
Scattering factor accuracy
1
Tile supercell x
4
Tile supercell y
4
Number of pixels in x
512
Number of pixels in y
512
<1> Continue <2> Change
1
Calculation type
1
<1> QEP <2> ABS
2
<1> Absorption <2> No absorption
1
Elastic and inelastic scattering output choice
1
<0> continue <1> beam tilt <2> specimen tilt
0
<0> continue <1> save <2> load
0
aperture cutoff
9.610000
Defocus
-560:0:280
Change image forming lens parameters
6
3rd order spher. coefficient
1.200000
Change image forming lens parameters
0
<0> Precalculated potentials <1> On-the-fly calculation
0

On another hand software runs fine for a very small calculation.

Only the following lines differ in HRTEM_example.txt for when software manages to run and when it fails:

For "success" (smaller) calculation, lines 21-28 are:
Tile supercell x
1
Tile supercell y
1
Number of pixels in x
128
Number of pixels in y
128

For "fail" (bigger) calculation, lines 21-28 are:
Tile supercell x
4
Tile supercell y
4
Number of pixels in x
512
Number of pixels in y
512

The difference is that for the bigger calculation, I am increasing the size of the super cell from 1 to 4 and I am increasing the number of pixels from 128 to 512.

Please advise.

Precession series results overwritten

Hi there,

I've recently started using the GPU double precision executable version of MuSTEM to simulate a precession series of CBED patterns. I have noticed that when I run this using absorption, this outputs the files with the azimuthal angles conveniently included in the filename to generate an easily identifiable series of files. However, when I use QEP instead, the output filename does not include this, and therefore each new pattern in the series overwrites the previous.

I'm happy to offer to look into this, however I won't have time to do so for a while.

Best,

Robert

v5.3 GPU memory check issue with QEP

Hamish - just an FYI, moving from v5.2 to v5.3 you added inputs to qep_stem_GPU_memory and qep_tem_GPU_memory in their respective multislice files; however, you did not include the inputs when calling the subroutines in qep_stem and qep_tem, respectively, causing a segmentation fault. After adding inputs to the call, everything seems to run smoothly.

Pixel size of CBED patterns

How can I know the pixel size of CBED patterns (dk) simulated by MuSTEM? For example, I divide a unit cell with lattice parameter a and b by Nx and Ny in the program, I found two ways to calculate dk and don't know which one is right.

  1. dk = min(1/a, 1/b)
  2. Use the does the 'Max scattering vectors' given by the program to calculate dk. Suppose the max scattering vectors are k_xmax and k_ymax, then dk = min(k_xmax/Nx, k_ymax/Ny)
    The two methods give different results. Could anyone tell me the real way the program uses to determine dk please?

Selecting start and stop position of probe in STEM/PACBED simulations

As far as I understand it is only possible to select the probe starting position in STEM/PACBED simulations and not the stop position? This option would be nice for e.g. simulating precipitates embedded in a matrix so that the tiling could be included in the .xtl-file. This would be less computationally heavy, since if one was to do the tiling in MuSTEM and on the same time account for wrap-around error, one would need a larger supercell than if one could account for both the tiling and the wrap-around error in the model itself.

Source code is not complete

Hi,

I am managing a high performance computing cluster at a research institution in US, and I am responsible for installing and maintaining scientific software that users use. I have several users request to install MuSTEM on the cluster. I have obtained 5.2 version from one of the users, and have it installed and run successfully. At this point, another user has requested a upgraded version, that is 5.3 version installed. I ended up pulling source files from this Github repository. In the "Source" folder, I notice that several f90 files are missing as it is complained by the compiler using the Makefile.
Two things I am not sure:

  1. Is the Makefile in this repository for 5.2 version or 5.3 version?
  2. Suppose the Makefile is right, where are the missing f90 files that are needed to compile the binary code in Linus OS?
    I appreciate your time and help!

How to calculate L2,3-edge at silicon?

I am using μSTEM (CPU version) to simulate silicon in the 011 incidence direction, and when performing EELS calculations, only the K,M-edge is displayed.

when we performed the M-edge simulation, the EELS map was localized to the atoms, so we assume that the L2,3-edge calculation is being performed.
      
In this case, how should we calculate the L2,3-edge?

 Ionization choices

Index  Atom| Z  | Orbital | Shell | Window (eV)| Included(y/n)
-----------------------------------------------------------
 < 1>  Si  | 14 |  1s     | K     |   0.0      | n
 < 2>  Si  | 14 |  3s     | M1    |   0.0      | n
 < 0> continue
 > 2
 Enter EELS energy window above threshold in eV (between 1 and 100 ev):

 > 30

Binary vs compiling source code

Hi Hamish,

Are there good reasons to compile MuSTEM oneself with regards to speed?
As in, do you think it run faster if I compile it, or is using the pre-compiled binary probably just as fast?

Compiling MuSTEM on Power9 architecture

I have access to an IBM Power9 GPU cluster. Obviously, the MKL libraries are not available for this architecture, but I was planning on trying to use the Makefile posted by @bryandesser. I'm relatively inexperienced when it comes to compiling things for clusters and fortran in general, but I was hoping to get this working on this system, since it has many powerful GPUs.

I'll update with how I make out, but I figured I should ask to see if there's any known limitations in the code that would prevent it from working without access to MKL.

Bug when performing line scan simulations

For testing purposes, I am trying to perform some line scan simulations. To do this, I change the sampling in the STEM parameters setup so that one of the dimensions is equal to 1. This works fine when setting the scan size in the X-dimension to 1 like so:

nxsample
    1
nysample
    16

However, if I do the opposite:

nxsample
    16
nysample
    1

the calculation seems to run to completion but the output is strange. Even though I selected two detectors, there is only a single output file and the name is non-sensical (??W??? with no extension).

Is there some reason for this behavior? Or, is this a bug?

New dll files missing

The latest version of the code doesn't appear to contain the .dll files within the executable folder. Unfortunately I also can't seem to find them lying around on the internet. Is it possible to upload a copy?

Can I use Linux version ?

Hello Dr. Hamish,

My name is Toshihiro Futazuka in The University of Tokyo, Japan.

I want to use muSTEM with linux cluster machine.
Can we compile source code in Linux?
Is there any Linux version of this software ?

I`m sorry for asking elementary question .......

Compatibility in Linux

Hello, I am a beginner in HAADF simulation and fortran coding. I am wondering if this code can be transplanted so that it can be runned in linux cluster. We hope to use it to simulate large system but our windows machine is not powerful enough. We are thinking to resort to our supercomputing nodes to fufill the problem.

Any binary in linux?

My question is on the line of previous ones about Linux system, can I have access to any MuSTEM executable already compiled for a Ubuntu installation?

Thank you for any guidance.

Juan

Save/load potentials

There appears to be a bug when attempting to save transmission functions for later use. In the load_save_add_grates interface (both *_abs and *_qep) at the end of each subroutine the grates are written out as follows (respectively):

write(3984) abs_grates*pi*spread(spread(a0_slice(3,1:n_slices),dim=1,ncopies=nopiy),dim=2,ncopies=nopix)/ak; close(3984

write(3984) qep_grates*pi*spread(spread(spread(a0_slice(3,1:n_slices),dim=1,ncopies=nopiy),dim=2,ncopies=nopix),dim=3,ncopies=n_qep_grates)/ak; close(3984)

and (as best as I can tell) the multiplication terms are unneccesary and the lines should be replaced with:

write(3984) abs_grates; close(3984)
write(3984) qep_grates; close(3984)

I tested this for ABS and QEP, TEM and STEM and now the results are exactly the same running the full computation or starting from precalculated transmission functions.

Unable to compile MS_utilities.f90

I'm trying to compile the CPU version of muSTEM using the community edition of PGI Community Edition v21.9. Everything seems to compile just fine except for MS_utilities.f90 where I get the following error:

NVFORTRAN-F-0000-Internal compiler error. Could not locate uplevel instance for stblock 1899 (MS_utilities.f90: 138)

I can't see anything in the code that should cause an error. Has this been observed in other instances? If so, what is the workaround?

Ionization Simulations missing necessary Folder?

Hi,

I've been trying to run the EDX/EELS STEM simulation available in the Tutorials directory for STO (STEM_SrTiO3_STEM_driver.txt) but it looks like the ionization data is missing in this repository.

Example Error: Problem reading ionization_data\EELS_EDX_1s.dat

I've checked all the directories and can't seem to find any of these files. Is this intentional and am I supposed to download these files elsewhere?

Play All not working

Hi Hamish, I tried to use the 'play all' option this afternoon

I hit an error when it tries to run the second file saying 'Array already allocated'.

cannot download the excutables

Dear Hamish
It does not seem that we can download these pre-excutable anymore? People would really appreciate it if they could be updated. PS: I am trying to use it for simulations as well. Thank you so much.
Best wishes
Kelvin

Memory allocation with PGI

I have found that certain newer versions of PGI throw memory allocation issues during the Pre-calculation setup stage when they encounter if(.not. ) statements that are not followed by then and a newline + endif. There are only a handful of these lines across the 4 simulation types (s_absorptive_tem.f90, s_absorptive_stem.f90, s_qep_tem.f90, s_qep_stem.f90). Seems to be a simple fix, for example:

Original: (118, s_absorptive_tem.f90)
if(.not.load_grates) projected_potential = make_absorptive_grates(nopiy,nopix,n_slices)

Fixed:

if(.not.load_grates) then 
projected_potential = make_absorptive_grates(nopiy,nopix,n_slices)
endif

I have seen this issue on PGI 18 and 19 community editions, on two HPC clusters (Ohio Supercomputer Center and M3 Massive) as well as a local workstation running Ubuntu. All were compiled with CUDA 9 or 10.

Compilation error

When I compile the binary using the source code, I get this error.
Did any one else get this error? How to solve the problem? Thanks!

ifort -mkl -fpp -c -qopenmp -recursive -Ddouble_precision -I /export/intel/compilers_and_libraries_2017.6.256/linux/mkl/include/fftw -I /export/intel/compilers_and_libraries_2017.6.256/linux/mkl/include/ -c m_potential.f90
m_potential.f90(775): error #6401: The attributes of this name conflict with those made accessible by a USE statement. [N_SLICES]
integer4,intent(in)::nopiy,nopix,n_slices
------------------------------------------^
m_potential.f90(775): error #6445: A dummy argument is required in this context. [N_SLICES]
integer
4,intent(in)::nopiy,nopix,n_slices
------------------------------------------^
m_potential.f90(791): error #6405: The same named entity from different modules and/or program units cannot be referenced. [N_SLICES]
do j = 1, n_slices
------------------^
m_potential.f90(792): error #6405: The same named entity from different modules and/or program units cannot be referenced. [N_SLICES]
write(,'(1x, a, a, a, a, a)') 'Calculating transmission functions for slice ', to_string(j), '/', to_string(n_slices), '...'
-----------------------------------------------------------------------------------------------------------------------------^
m_potential.f90(775): error #6279: A specification expression object must be a dummy argument, a COMMON block object, or an object accessible through host or use association. [N_SLICES]
integer
4,intent(in)::nopiy,nopix,n_slices
------------------------------------------^
m_potential.f90(763): error #6404: This name does not have a type, and must have an explicit type. [N_SLICES]
function make_absorptive_grates(nopiy,nopix,n_slices) result(projected_potential)
------------------------------------------------^
compilation aborted for m_potential.f90 (code 1)
make: *** [m_potential.o] Error 1
[root@mgt2 CPU_routines]# make all
ifort -mkl -fpp -c -Ddouble_precision -I /export/intel/compilers_and_libraries_2017.6.256/linux/mkl/include/fftw -I /export/intel/compilers_and_libraries_2017.6.256/linux/mkl/include/ -c m_potential.f90
m_potential.f90(775): error #6401: The attributes of this name conflict with those made accessible by a USE statement. [N_SLICES]
integer4,intent(in)::nopiy,nopix,n_slices
------------------------------------------^
m_potential.f90(775): error #6445: A dummy argument is required in this context. [N_SLICES]
integer
4,intent(in)::nopiy,nopix,n_slices
------------------------------------------^
m_potential.f90(791): error #6405: The same named entity from different modules and/or program units cannot be referenced. [N_SLICES]
do j = 1, n_slices
------------------^
m_potential.f90(792): error #6405: The same named entity from different modules and/or program units cannot be referenced. [N_SLICES]
write(,'(1x, a, a, a, a, a)') 'Calculating transmission functions for slice ', to_string(j), '/', to_string(n_slices), '...'
-----------------------------------------------------------------------------------------------------------------------------^
m_potential.f90(775): error #6279: A specification expression object must be a dummy argument, a COMMON block object, or an object accessible through host or use association. [N_SLICES]
integer
4,intent(in)::nopiy,nopix,n_slices
------------------------------------------^
m_potential.f90(763): error #6404: This name does not have a type, and must have an explicit type. [N_SLICES]
function make_absorptive_grates(nopiy,nopix,n_slices) result(projected_potential)
------------------------------------------------^
compilation aborted for m_potential.f90 (code 1)
make: *** [m_potential.o] Error 1

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.