The University of Arizona
SyMAP Trouble Shoot  
AGCoL | SyMAP Home | Index | System Guide | User Guide | Tour | Troubleshooting
Please send questions and suggestions to symap@agcol.arizona.edu.

As of SyMAP v5.0.6, the Applet has been removed and will no longer be supported; it is in earlier SyMAP versions, and can be obtained at github.

Contents


Memory

Not enough memory

For very large projects, or queries involving multiple genomes (e.g. 5 or more), the Project Manager may run out of memory. To fix this problem, run symap with the -m parameter to increase the maximum memory used from the default 2048 MB to 4096 MB.:
     > ./symap -m 4096
Alternatively, you can edit the file symap and change
my $maxmem="2048m"; # change this line to use "4096m" 
You will also need to be using a 64-bit processor.

Return to top


MySQL suggested settings

Slow to load sequence and annotation

Two MySQL settings are especially important for SyMAP performance (and generally for InnoDB table performance).

innodb_buffer_pool_size The default is too low for most purposes. You should set this to around 1Gb if possible (note that the units are bytes).
innodb_flush_log_at_trx_commit=0 The default setting is 1, which results in very slow uploading of data.

To check the values, start mysql and type:

show variables like "innodb_buffer_pool_size"; 
show variables like "innodb_flush_log_at_trx_commit"; 
To change the variables, then type:
set global innodb_buffer_pool_size=1073741824;
set global innodb_flush_log_at_trx_commit=0;
Alternatively, You can set these in the MySQL configuration file my.cnf and restart MySQL. Note that my.cnf is typically located at /etc/my.cnf, on both Linux and Mac.

Different machines and MySQL installations can produce different results with these variables. If the SyMAP demo-seq to demo-seq2 load and synteny computations seem slow, try different combinations of these two variables to see what performs the fastest inserts.

Slow queries

max_allowed_packet This should be at least 500M if possible (the units are bytes).

To check the value, start mysql and type:

show variables like "max_allowed_packet"; 
To change this variable, then type:
set global max_allowed_packet=1073741824;

Return to top


MySQL

Database connect errors

After startup, if the program is left idle for some time then its database connection will close and it will show a complicated-looking error on next usage, or it may say this:
com.mysql.jdbc.CommunicationsException: Communications link failure due to underlying exception:

** BEGIN NEXTED EXCEPTION **
In this case the solution is to restart the program.

On startup, error messages mentioning the database or SQL probably mean that the database connection could not be established or was lost. A message at startup means SyMAP was not able to connect to the database using the provided information in symap.config. First, make sure your db_adminuser and db_adminpasswd are correct.

These days, the best way to find the MySQL problems is to search the internet. However, the following are some things to check for connection problems on SyMAP startup:

Check the database privileges: The database users specified in symap.config may not have necessary access to the database. If you running the read/write symap, then the relevant user is db_adminuser. If you are running a read-only session using viewSymap, it will use the read-only db_clientuser or read-write db-adminuser. In either case the user needs to have database access privileges from the machine where you are working. Note that MySQL privileges are granted specifically to certain users on certain machines. You can test the user access by using the MySQL command line client mysql, if it is installed on your machine, or by looking at the privilege table using an administration interface such as PHPMyAdmin. The mysqlaccess tool is also useful, if it is installed on your machine.

Check the MySQL configuration: The MySQL configuration file (/etc/my.cnf) should not contain any of the following lines that prevent remote access to the database.

     
	bind_address=127.0.0.1
	skip_networking 

Make sure the port is visible: If the database is on a different computer, test that its port 3306 is visible from your computer:

     telnet <server address> 3306
If it does not make a connection then either the server is not running, or it is set to run on a non-standard port, or the port is blocked by a firewall. Contact a system administrator. (Note, to get out of telnet type "^]".)

MariaDB cannot connect

If you are using MariaDB (post version 10.4.12) and get the error:
Unable to connect to the mysql database on localhost; are username and password correct?
Try adding the following lines to the /etc/my.cnf:
[mysqld]
character-set-server=utf8mb4
And restart MariaDB with systemctl restart mariadb.

Thanks to Lori for this solution!

The database does not support Innodb tables

This can happen in a couple of ways ways. Look in the mysql log for clues.
  • MySQL started with the "skip-innodb" flag. Solution is to remove this.
  • InnoDB log file problems. This can happen if you change the log file size, without deleting the log file. Solution: shut down mysql, remove the innodb log file, and restart mysql.

    Please read "Security" section of the manual to find out how to run mysqld as root

    You are launching SyMAP as the unix root user, which is blocked by MySQL as a security hazard. Solution is simply to use a regular non-root user account.

    Can't create table (errno: 121)

    This can occur if you try to create a SyMAP database whose name differs only by case from an existing database, e.g. "symapTest" when "symaptest" already exists.

    Return to top


    External programs

    The SyMAP package comes with compiled versions of MUMmer (sequence alignment), Blat (FPC alignment) and Muscle (SyMAP Queries). These programs are in the symap_5/ext directory. There are compiled versions for 64-bit Linux and 64-bit MacOS. For example, in the symap_5/ext/mummer directory, there are sub-directories linux64 and mac. SyMAP will select the correct directory for the machine you are running from, i.e. you do not need to do anything.

    Not executable

    When SyMAP creates a database, it (1) checks the MySQL variables, and (2) checks that the external programs are executable. If you see a message like:
    ***Error - file is not executable: ext/mummer/mac/promer
    
    Execute:
    > chmod 755 ext/mummer/mac/promer
    

    MacOS 10.15 (and possibly earlier versions)

    Any executable that has not been okayed by Apple get the error message "cannot be opened because the developer cannot be verified". Hence, you need to use the Finder to enter all /ext/<program>/mac directories, select each program, then choose to Open it; it will run the program in a terminal window and be saved as a program that can thereafter be run.

    Fails to execute

    If when "Alignment&Synteny" is executing, you get a message to the terminal like:
    Error running command: /Users/cari/Workspace/github/symap/ext/mummer/mac/promer  
    -p data/seq_results/demo_seq_to_demo_seq2/align/demo_seq_cc.demo_seq2_f2.promer 
    data/seq_results/demo_seq_to_demo_seq2/tmp/demo_seq2/demo_seq2_f2.fa 
    data/seq_results/demo_seq_to_demo_seq2/tmp/demo_seq/demo_seq_cc.fa
    
    One of the following should elucidate the problem (say the problem occurred when running promer on MacOS):
    1. Enter the logs directory, then the directory of the pair (e.g. demo_seq_to_demo_seq2). View one of the .log file, which contains the output of promer.
    2. Copy the command (beside "Error running command:") from the terminal, and past it on a new terminal line to execute. This shows promer output directly on the terminal.
    3. Enter the ext/mummer/mac directory (or the appropriate one), and execute the command at the command line, e.g ./promer.
    All the above give additional that may show the problem and lead to the fix.

    Otherwise, you may need to compile the program on your machine. In this case, replace the existing executables with your compiled ones. For example, if you compile MUMmer, look at the respective ext/mummer/mac directory and copy the corresponding programs in. The promer and nucmer are perl scripts that have paths hard-coded at the top, which need to be changed to ./ext/mummer/mac/.

    Early versions of MacOS

    The MacOS executables were compiled on a MacOS 10.15. These will not work on old versions such as MacOS 10.9; however, the directories with the the pre506 suffixes. If you do the following:
    mv mac mac_506
    mv mac_pre506 mac
    
    These executables compiled ona MacOS 10.9 may work on your Mac.

    32-bit

    It seems that 32-bit machines are history. However, the linux directories have executables assembled on a 32-bit machine.

    Return to top


    Alignment

    If MUMmer or Blat fail to run, see the above section.

    Status window disappeared

    Occasionally the status window disappears when running an alignment. Clicking the top of the project manager sometimes brings it back. Otherwise, just check the output to the terminal, which tells you when it is done. You may have to restart symap to see the results.

    MUMmer failed

    Try using a hard-masked genome sequence. Ensembl supplies both soft-masked and hard-masked genome sequences. NCBI supplies soft-masked sequences, where the supplied ConvertNCBI script (scripts/ConvertNCBI) can convert the file to hard-mask.

    MUMmer/PROmer cannot handle large sequences

    If the alignment doesn't produce results and you see errors indicating that PROmer cannot produce the .aaqry file, then the mummer binary may not be 64-bit. You can tell for sure by looking at the output lines like
    # (maximum reference length is 536870908)
    # (maximum query length is 4294967295)
    
    If you see small numbers like this, it is not 64-bit. For 64-bit, the numbers are extremely large, much larger than any possible genome. To build a 64-bit mummer, you must set
    #define SIXTYFOURBITS 
    
    in src/kurtz/libbasedir/types.h (and, of course, compile on a 64-bit machine). We fixed this problem in the 3.4 version of SyMAP, for lintel64, but if you're building for a different platform you'll have to make this fix.

    No anchors loaded

    This message indicates that SyMAP could not parse the MUMmer or BLAT output files to load the anchors.

    The most likely reason is that MUMmer or BLAT failed to run. Usually this is because the platform (linux 32/64 bit, or mac) was not detected properly. First try re-starting symap using the '-32, -64, -linux, -mac' parameters to specify your platform (run 'symap -h' for list of parameters).

    The next likely cause is trying to run a whole-chromosome MUMmer alignment on a 32-bit machine. MUMmer usually uses more than 2Gb per thread, which is not possible on a 32-bit machine. If you suspect this is the problem, try running on a 64-bit machine. If you compiled MUMmer yourself, check the above section.

    If that does not resolve the problem, The next step is to look in the project .log file in the /logs directory for more detailed error messages. Then, you should look at the MUMmer log files (extension .mum.log) in the directory

    logs/<proj1_to_proj2>
    
    These contain the messages printed by MUMmer, and should show why it failed to run.

    FPC: BES not loaded, or BES hits not loading

    Generally, this is caused by a name formatting error. Make sure the BES names are formatted as clone names with extension "r" or "f". The clone names should exactly match those in FPC, and there should be no other prefix or suffix. Do not use ".r" or ".f".

    Return to top

     

  • Email Comments To: symap@agcol.arizona.edu