Difference between revisions of "Solution to App Catalog Installation Limit"

From WebOS Internals
Jump to navigation Jump to search
 
(58 intermediate revisions by one other user not shown)
Line 1: Line 1:
Based on rwhitby's findings that temporarily moving /var/usr/palm/applications will allow installing apps from the App Catalog, here's a permanent way to keep the apps on (much larger) /media/internal by using links. This will permanently save space on /var.
+
== Overview ==
 +
The 'mvapp' method is a Linux command that allows moving your Pre apps and email to the (much larger) media drive by using symbolic links. This will save space on internal system storage (/var) and allow more App Catalog apps to be installed, up to media drive limit (gigabytes) rather than the 64MB limit.
  
It's unfortunate that Palm has not resolved the issue in the 1.2 update. This is a tactical workaround that could potentially work long term, also saving space on /var, but this needs support by the homebrew community to add in the homebrew installer apps.
+
This method involves creating a (hidden) directory /media/internal/.apps, moving selected applications to the newly created directory, and then creating a symbolic link in the /var/usr/palm/applications directory pointing to the new location.  mvapp v0.3.0 and higher now also moves email and attachments to /media/internal/.data. This frees up the disk space from the relatively limited internal system volume (/var) to the much larger media drive (/media/internal). The 'mvapp' command (available on PreWare) will move the files and create the appropriate link.
 
 
== Overview ==
 
This solution involves creating a (hidden) directory in the /media/internal area, moving selected applications to the newly created directory, and then creating a symbolic link in the /var/usr/palm/applications directory pointing to the new location. This frees up the disk space from the relatively limited /var volume to the larger /media/internal. The included script (which must be created) will move the files and create the appropriate link. It will also provide information on the size of the applications stored in the /var/usr/palm/applications directory.
 
  
 
== Solution (symbolic link method) ==
 
== Solution (symbolic link method) ==
  
Create a script to move apps to new home and create link. You can use an editor or use the following cat command to paste the code, for those not familiar with editors.
+
'''Install mvapp''':<br>
 
+
Now on PreWare in the Linux Applications category. In PreWare Preferences, need to have 'show available types' turned on.
Here's how to do it for those comfortable with accessing Linux...
 
  
- Enter Linux mode on Pre through a computer (linux commands are in bold)
 
 
- Set read/write to allow creating script<br>
 
<code>'''mount -o remount,rw / '''</code>
 
 
- create a script using cat command, paste the code below and then do 'ctrl c' on the keyboard to break out.<br>
 
<code>'''cat >/usr/bin/mvapp''' </code>
 
 
- paste the code below here, press ctrl-c on keyboard to exit after pasting the code.
 
 
- when you see a linux prompt, set command to be executable<br>
 
<code> '''chmod 755 /usr/bin/mvapp''' </code>
 
 
----
 
code:
 
 
<pre>
 
<pre>
#!/bin/sh
+
Quick Start Guide
 +
- Launch PreWare
 +
- In PreWare Preferences, need to have 'Show All Package Types' turned on
 +
- Install 'mvapp' from PreWare in the Available Other -> Linux Application category
 +
- Install Terminal app from PreWare (if not already installed)
 +
- Open Terminal from Launcher, then type...
  
# This code is open for re-use with no restrictions.  xorg
+
mvapp doctor
# This is a working proof of concept script still in development.
+
mvapp bulkmv
# Intent is for someone to port to or use this for a webOS app.
+
</pre>
# Use at your own risk.
 
  
#-------------------------------------------------------------------------#
+
== Prepping for webOS 1.3.5 Update ==
# versions:
+
There's a pretty good chance the next webOS update will address the app limit storage issues, which will mean 'mvapp' will likely not be needed from that point on.
# 0.1.0 - original (xorg)
 
# 0.1.1 - added unlink and clean functions (daventx)
 
# 0.1.2 - added bulkmv function, allows moving many apps (xorg)
 
# 0.1.3 - added option for tar backups (xorg)
 
# 0.1.4 - added listmoved function to show apps already moved (xorg)
 
# 0.1.5 - added restoreall function, couple cleanup items (xorg)
 
# 0.1.6 - fixed to show usage if no appname supplied to link/unlink (xorg)
 
# 0.1.7 - added cleanexit (w/mount ro /)
 
#      - added exit code documentation for javascripts calling this (xorg)
 
# 0.2.0 - will not move apps that would have attribute issues (xorg)
 
#      - will not move apps that have no json file
 
#      - improved error handling, improved listing shows actual app names
 
# 0.2.1 - minor update, proper cleanup if link command doesn't move app
 
#      - don't create tar backup until testing if app is movable (xorg)
 
# 0.2.2 - broke with Fair Dinkum - resolved by explicit path for du (nt4cats)
 
#      - will no longer move an app if it is running, thx to greg roll
 
#      - faster listmoved apps (but no size), error reporting cleanup (xorg)
 
# 0.2.3 - new trap method for link cp errors ; version reporting in usage (xorg)
 
#-------------------------------------------------------------------------#
 
  
#-------------------------------------------------------------------------#
+
Best Case Scenario:
# variables: these are globally available to all functions
+
There is the possibility that mvapp will be compatible with 1.3.5. It may be able to maintain the links with the new app storage method. If the case, you can do nothing or 'mvapp restoreall' after the update. I'll likely able to answer this before 1.3.5 is released.
#-------------------------------------------------------------------------#
 
COMMAND=$1
 
APP=$2
 
VERSION=0.2.3
 
MEDIA=/media/internal/.apps
 
VAR=/var/usr/palm/applications
 
  
# Backup keeps file attributes but uses more /media space. Recommend doing backups.
+
Otherwise...
BACKUP=1  # set to 1 for tar backups, 0 to disable
 
BACKUPDIR=/media/internal/.appbackups
 
  
# This should be turned on. Only turn off if javascript is calling this script.
+
Pre Update Option:
PROMPTS=1
+
In preparation, just before upgrading to webOS 1.3.5, you can perform a 'mvapp restoreall' to restore apps and 'mvapp unlinkemail' to restore email back to the /var location.
  
#-------------------------------------------------------------------------#
+
However if you have accumulated dozens of apps and/or a large amount of email, you may not be able to restore fully and have enough room for the upgrade. You may need to do some cleanup by removing apps that you do not use. After the update, you should be able to reinstall your deleted apps. If willing, you may also consider deleting your email accounts and reinstalling before performing 'mvapp unlinkemail'. Though I suspect that linked email will be compatible with 1.3.5.
# exit codes for javascripts:
 
# function usage:
 
# 1 - normal usage error
 
#
 
# function cleanapp:
 
# 0 - normal exit
 
#
 
# function linkapp:
 
# 10 - app name not supplied
 
# 11 - link already exists
 
# 12 - app does not exit in VAR
 
# 13 - copy failed from VAR to MEDIA
 
# 14 - removing app from VAR failed
 
# 15 - APP needs permissions not supported on FAT, did not move to MEDIA
 
# 16 - APP has no json file, did not move to MEDIA
 
# 17 - APP is running, need to close the app before moving
 
#
 
# function unlinkapp:
 
# 20 - app name not supplied
 
# 21 - app doesn't exit on MEDIA
 
# 22 - tar restore failed
 
# 23 - copy failed
 
# 24 - remove failed
 
#
 
#-------------------------------------------------------------------------#
 
  
#-------------------------------------------------------------------------#
+
Post Update Option:
# function: cleanexit - exit with cleanup items
+
I'm planning to have a secondary recovery option for those who perform an OS update before getting a chance to restore mvapp links, but it won't be released until the day of the update. Hopefully 1.3.5 will be compatible with existing mvapp links and no preparation will be needed.
#-------------------------------------------------------------------------#
 
  
cleanexit () {
+
Follow this section for more details.
code=$1
 
  
# put / back to read only
+
== Precautions ==
  mount -o remount,ro /
 
  
# Uncomment if you want verbose exit codes
+
So far, no one has reported issues with apps or email due to linking.  These are best practices to avoid issues.
# echo "exit code: $code"
 
  
exit $code
+
- Version 0.3.0 and higher now can move email and attachments to media drive. It is unknown how a future webOS release will deal with the storage issues. It's possible a future webOS release could break the email link to the media drive. While I'd likely have a new mvapp release out before next webOS release, losing email on the phone is a possibility and you may have to resetup email accounts. Moving email is not recommended if you rely on email for your job unless you don't mind possibly losing all email on the phone. Your email will still exist at the original source (gmail, yahoo, hotmail, etc), so linking email is not likely to have much of an impact for most of us.
}
 
# end of cleanexit function
 
  
 +
- If Palm releases a new webOS that resolves the overall issue, you _may_ (but not likely) need to completely restore your apps back to /var before performing the upgrade - mvapp restoreall. For email - mvapp unlinkemail. If a future webOS update is applied before you get a chance to restore, it's possible it could break this method, so moving apps that store important information to you is not recommended. Since it would likely be a significant architectural change for Palm to fix this issue, storage issues may not be officially resolved until webOS 2.0, but who knows. webOS 1.3.1 does not have any issues with mvapp.
  
#-------------------------------------------------------------------------#
+
- This is a workaround script not approved by Palm. The usual disclaimers apply.
# function: usage - show command usage options
 
#-------------------------------------------------------------------------#
 
usage () {
 
  exitcode=$1
 
  if [ ! $exitcode ]
 
  then
 
    exitcode=1 # default exit code for usage, otherwise exit with incode
 
  fi
 
  
  echo "mvapp version: $VERSION"
+
== Commands ==
  echo "Usage: mvapp link domain.appname    - move app to media and link"
+
'''To run diagnostics on /var and get recommendations'''
  echo "Usage: mvapp unlink domain.appname  - restore app to var, remove link"
 
  echo "Usage: mvapp clean domain.appname  - remove app dir and links"
 
  echo "Usage: mvapp list                  - list all apps sorted by size"
 
  echo "Usage: mvapp bulkmv                - move/link bulk apps"
 
  echo "Usage: mvapp listmoved              - list apps that have been moved"
 
  echo "Usage: mvapp restoreall            - restore all apps to original"
 
  
  cleanexit $exitcode
+
<code>mvapp diag<br>
}
+
(or shortcut)<br>
# end of usage function
+
m d<br>
 +
m d m (includes media apps, slower)</code>
  
 +
This reports the usage of applications, email and attachments on /var.  It also recommends actions based on the results.
  
#-------------------------------------------------------------------------#
 
# function: cleanapp - removes symbolic links and folder in media and var
 
#-------------------------------------------------------------------------#
 
  
cleanapp () {
+
'''To find the largest apps in /var/usr/palm/applications'''
  
mount -o remount,rw /
+
<code>mvapp list<br>
 +
(or shortcut)<br>
 +
m ls</code>
  
# exit to usage if no app name supplied
+
The first number is size of app in KB, largest shown last. IE....<br>
if [ ! $APP ]
+
<code>
then
+
8352 com.apnews.webos<br>
  usage 1
+
8512 com.fandango.app.fandango<br>
fi
+
8672 com.palm.app.musicplayerremix<br>
 +
10304 com.shortcovers.palm.pre<br>
 +
10432 com.fusioncreativestudios.deadman<br>
 +
10656 com.ulocate.app.where<br>
 +
</code>
  
# Continue on if PROMPTS turned off (call from outside app)
 
if [ $PROMPTS ] 
 
then
 
  echo "This will remove $APP from both $VAR and $MEDIA."
 
  echo "You should first attempt to remove the app using the official Pre methods."
 
  echo "Are you sure you want to remove $APP? [y/N]: "
 
  read answer
 
  case $answer in
 
  [Yy]*) continue;;
 
      *) cleanexit 0;;
 
  esac
 
fi
 
  
if [ -h $VAR/$APP ]
+
'''To move and link an app to /media'''
then
 
  echo "Removing all traces of $APP."
 
else
 
  echo "$APP does not exist..."
 
  usage 10
 
fi
 
  
echo "Size of $VAR before cleanup... "
+
<code>mvapp link domain.appname<br>
/usr/bin/du -sh $VAR
+
(or shortcut)<br>
 +
m ln domain.appname</code>
  
  if [ -d $MEDIA/$APP ]
+
Example: mvapp link com.ulocate.app.where
then
 
    rm -r $MEDIA/$APP
 
      echo "Removed directory" $MEDIA/$APP
 
fi
 
if [ -d $VAR/$APP ]
 
then
 
      rm -r $VAR/$APP
 
      echo "Removed directory" $VAR/$APP
 
fi
 
if [ -L $VAR/$APP ]
 
then
 
      rm -r $VAR/$APP
 
      echo "Removed link" $VAR/$APP
 
fi
 
if [ -f $BACKUPDIR/$APP.tgz ]
 
then
 
      rm -r $BACKUPDIR/$APP.tgz
 
      echo "Removed tar backup" $BACKUPDIR/$APP.tgz
 
fi
 
  
 +
The app should now work in the new location thanks to the link. Test each app to make sure it works before doing another.
  
# rescan luna in case it's needed
+
Continue moving apps until it reports the 'var used' is below 100MBIf 'var used' is 95MB, you could add 5MB more apps.
luna-send -n 1 palm://com.palm.applicationManager/rescan {} >/dev/null 2>&1
 
  echo "$APP directories and links removed."
 
echo "Size of $VAR after cleanup... "
 
/usr/bin/du -sh $VAR
 
cleanexit 0
 
}
 
# end of cleanup function
 
  
  
#-------------------------------------------------------------------------#
+
'''To move and link many apps'''
# function: listapps - list the size of each app, sort showing largest last
 
#-------------------------------------------------------------------------#
 
listapps () {
 
echo "First number is size in KB.  Size of 0 is an app already linked."
 
cd $VAR
 
for i in `/usr/bin/du -s * | sort -n |cut -f 2`
 
do
 
    APP=$i
 
    SIZE=`/usr/bin/du -s $VAR/$APP |cut -f1`
 
    TITLE=""
 
    if [ -f $VAR/$APP/appinfo.json ]
 
    then
 
      TITLE=`grep title $VAR/$APP/appinfo.json |cut -d: -f2 |cut -d\" -f2`
 
    fi
 
    echo "$SIZE - $APP - $TITLE"
 
done
 
 
cleanexit 0
 
}
 
# end of listapps function
 
  
#-------------------------------------------------------------------------#
+
<code>mvapp bulkmv<br>
# function: listmoved - list apps moved/linked, sort showing largest last
+
(or shortcut)<br>
#-------------------------------------------------------------------------#
+
m bm</code>
listmoved () {
 
#/usr/bin/du -sk $MEDIA/* | sort -n  #doesn't show proper size on FAT fs, removing
 
cd $MEDIA
 
#for i in `/usr/bin/du -s * | sort -n |cut -f 2`
 
for i in `ls $MEDIA`
 
do
 
    APP=$i
 
    # Not sure why du reports incorrectly on FAT fs
 
    #SIZE=`/usr/bin/du -s $MEDIA/$APP |cut -f1`
 
    SIZE=""
 
    TITLE=""
 
    if [ -f $MEDIA/$APP/appinfo.json ]
 
    then
 
      TITLE=`grep title $MEDIA/$APP/appinfo.json |cut -d: -f2 |cut -d\" -f2`
 
    fi
 
    echo "$SIZE - $APP - $TITLE"
 
done
 
cleanexit 0
 
}
 
# end of listmoved function
 
  
 +
This will show largest app first, ask if you want to move/link and then moves on to the next largest app.  If you answer no to an app, it will skip to the next.  Answer 'q' to quit.  Is easiest to use this method directly on the phone using the Terminal app, available in Homebrew.
  
#-------------------------------------------------------------------------#
+
'''To search for apps by partial app name'''
# function: linkapp - move the app to media and create symbolic link
 
#-------------------------------------------------------------------------#
 
linkapp () {
 
  
if [ ! $APP ]
+
<code>mvapp search<br>
then
+
(or shortcut)<br>
      echo "No application supplied..."
+
m s</code>
      usage 10
 
fi
 
 
# Check if the app is running
 
luna-send -n 1 palm://com.palm.applicationManager/running {} 2>&1 | grep $APP >/dev/null 2>&1
 
if [ $? == 0 ]
 
then
 
  echo "$APP is running.  Close the app before moving."
 
 
 
  code=17
 
  return $code
 
 
 
fi 
 
 
 
 
if [ ! -f $VAR/$APP/appinfo.json ]
 
then
 
    echo "$APP has no json file.  Will not be moved."
 
    code=16
 
    return 16
 
fi 
 
  
  if [ ! -d $MEDIA ]
+
This is an easier way to link/unlink/clean apps from the Terminal app. You only need to supply a partial app package name. It will show any matches then ask what you'd like to do with it.
  then
 
      mkdir $MEDIA
 
fi
 
  
if [ -h $VAR/$APP ]
+
'''To restore the app to original location and delete the link'''
then
 
      echo "Link already exists for... ${APP}"
 
      cleanexit 11
 
fi
 
  
TITLE=`grep title $VAR/$APP/appinfo.json |cut -d: -f2 |cut -d\" -f2`
+
<code>mvapp unlink domain.appname<br>
+
(or shortcut)<br>
mount -o remount,rw /
+
m ul domain.appname</code>
  
if [ -d $VAR/$APP ]
+
The application will be moved back to the original directoryIf you have issues with an app after unlinking, see the Contingency Plan section below.
then
 
  echo "Moving $APP $TITLE to $MEDIA..."
 
  else
 
  echo "$APP does not exist..."
 
  usage 12
 
fi
 
  
 
mount -o remount,rw /
 
  
echo "Size of $VAR before move... "
+
'''To list apps that have been moved and linked...'''
/usr/bin/du -sh $VAR
 
  
# move over to USB drive
+
<code>mvapp listmoved<br>
cp -rp  $VAR/$APP $MEDIA >/tmp/cpresult.out 2>&1
+
(or shortcut)<br>
if [ -s /tmp/cpresult.out ]
+
m lm</code>
then
 
  grep "cannot preserve" /tmp/cpresult.out >/dev/null 2>&1
 
  if [ $? = 0 ]
 
  then
 
  echo
 
  echo "$APP cannot be moved as it contains special attributes."
 
  echo "Leaving app in $VAR."
 
  code=15
 
  rm -r $MEDIA/$APP
 
  rm /tmp/cpresult.out
 
  return $code
 
  else
 
  echo "Copy failed. Leaving app in $VAR."
 
  code=13
 
  fi
 
  rm -r $MEDIA/$APP
 
  rm /tmp/cpresult.out
 
  return $code
 
fi
 
                 
 
  
# Backup using tar if enabled
+
This shows a list of apps that have been moved and linked to /media.   
if [ $BACKUP ]
 
then
 
  if [ ! -d $BACKUPDIR ]
 
  then
 
    mkdir $BACKUPDIR
 
  fi
 
  echo "Backing up $APP $TITLE to $BACKUPDIR..."
 
  tar czf $BACKUPDIR/${APP}.tgz $VAR/$APP
 
  fi
 
  
rm -r $VAR/$APP
 
if [ $? != 0 ]
 
then
 
  echo "Remove failed. Leaving app in $VAR."
 
  rm -r $MEDIA/$APP
 
  cleanexit 14
 
fi
 
  
# create the symbolic link
+
'''To restore all moved apps back to original location'''
ln -s $MEDIA/$APP $VAR/$APP
 
  
# rescan luna in case it's needed
+
<code>mvapp restoreall<br>
luna-send -n 1 palm://com.palm.applicationManager/rescan {} >/dev/null 2>&1
+
(or shortcut)<br>
 +
m ra</code>
  
echo "$APP SUCCESSFULLY moved and linked."
+
This will restore ALL applications that have been moved/linked, back to the original location in /varThis may take several minutes to complete. It will show each application progress. Beware that if you have moved a very large number of apps over a period of time, they may not all fit on /varYou may need to remove some/many unused apps before restoring all.
  echo "Size of $VAR after move... "
 
  /usr/bin/du -sh $VAR
 
}
 
# end of linkapp function
 
  
  
#-------------------------------------------------------------------------#
+
'''To cleanup and remove directories and symlinks'''
# function: unlinkapp -  restore the app to var and remove symbolic link
 
#-------------------------------------------------------------------------#
 
unlinkapp () {
 
  
if [ ! $APP ]
+
<code>mvapp clean domain.appname<br>
then
+
(or shortcut)<br>
  echo "No application supplied..."
+
m c domain.appname</code>
  usage 20
 
fi
 
  
  mount -o remount,rw /
+
If you have any issues, first attempt to remove the app from Launcher, the App Catalog or Homebrew app installer.  It's very important you remove from the phone first before using the clean command.  Then issue the 'clean' command to remove the applications directories at both locations and also the symlink. You can then reinstall the app from the App Catalog or any Homebrew App installer.
  
if [ -d $MEDIA/$APP ]
 
then
 
  echo "Restoring $APP..."
 
else
 
  echo "$APP does not exist..."
 
  usage 21
 
fi
 
  
echo "Size of $VAR before move... "
+
'''To move and link email and attachments'''
/usr/bin/du -sh $VAR
 
  
# remove the old symbolic link
+
<code>mvapp linkemail<br>
rm -r $VAR/$APP
+
(or shortcut)<br>
 +
m le</code>
  
# move to original location or restore from tar if it exists
+
This will move all of your email from /var to /media/internal/.data/emailsThis will close the email application so if you are writing a new email, be sure to send the email before using this commend. Also see the Precautions section above.
if [ -f $BACKUPDIR/$APP.tgz ]
 
then
 
  cd /
 
  tar xzf $BACKUPDIR/$APP.tgz
 
  if [ $? != 0 ]
 
then
 
  echo "Tar restore failed. Remove and restore app using official webOS/Pre methods."
 
  cleanexit 22
 
  else
 
  rm -r $BACKUPDIR/$APP.tgz
 
  fi
 
  else
 
  cp -r  $MEDIA/$APP $VAR
 
  if [ $? != 0 ]
 
  then
 
  echo "Copy failed. Leaving app in $MEDIA."
 
  cleanexit 23
 
  fi
 
fi
 
  
rm -r $MEDIA/$APP
+
'''To restore and unlink email and attachments'''
if [ $? != 0 ]
 
then
 
  echo "Remove failed. Leaving app in $MEDIA."
 
  rm -r $VAR/$APP
 
  cleanexit 24
 
fi
 
  
# rescan luna in case it's needed
+
<code>mvapp unlinkemail<br>
luna-send -n 1 palm://com.palm.applicationManager/rescan {} >/dev/null 2>&1
+
(or shortcut)<br>
 +
m ue</code>
  
echo "$APP moved and unlinked."
+
This will restore all of your email from /media back to /varThis will close the email application so if you are writing a new email, be sure to send the email before using this commend.
  echo "Size of $VAR after move... "
 
/usr/bin/du -sh $VAR
 
}
 
# end of unlinkapp function
 
  
#-------------------------------------------------------------------------#
+
== Hidden Commands ==
# function: bulkmv -  move/link many apps
 
#-------------------------------------------------------------------------#
 
bulkmv() {
 
echo
 
echo
 
echo "This allows moving many apps, asking which you'd like to move."
 
echo "Starting with the largest apps."
 
echo
 
  
 +
These commands are not documented in the Usage display.
  
mount -o remount,rw /
+
'''Check Links'''
cd $VAR
 
  
for i in `/usr/bin/du -s * | sort -nr |cut -f 2`
+
<code>mvapp checklinks<br>
do
+
(or shortcut)<br>
    export APP=$i
+
m cl</code>
    SIZE=`/usr/bin/du -sh $APP |cut -f 1`
 
    TITLE=`grep title $VAR/$APP/appinfo.json 2>/dev/null |cut -d: -f2 |cut -d\" -f2`
 
    echo
 
    echo
 
    echo "Size of $APP - $TITLE is $SIZE."
 
    echo "Would you like to move and link... $TITLE? [y/N/q]: "
 
    read answer
 
    case $answer in
 
    [Yy]*) linkapp;;
 
    [Qq]*) cleanexit 0;;
 
        *) echo "$APP not moved."
 
          continue;;
 
    esac
 
  
    echo
+
Checks if apps in /media have a link. If not, you have the option to remove the app from media, which is recommended.  This can occur if the app was not fully removed when you removed from Launcher or homebrew installer.
  done
 
}
 
# end of bulkmv function
 
  
#-------------------------------------------------------------------------#
+
'''Check if Fair Dinkum is installed'''
# function: restoreall - restore all apps, back to /var
 
#-------------------------------------------------------------------------#
 
restoreall() {
 
  
#Only confirm if PROMPT turned on. (allows outside app to call)  
+
<code>mvapp checkfd<br>
  if [ $PROMPT ]
+
(or shortcut)<br>
  then
+
m cfd</code>
  echo "This will restore all applications back to original location"
 
  echo "and remove the links.  Are you sure you want to continue? [y/N]:"
 
  read answer
 
  case $answer in
 
    [Yy]*) continue;;
 
        *) cleanexit 0;;
 
  esac
 
  fi
 
  
  ls $MEDIA | while read APP
+
Checks if Fair Dinkum is/isn't installed and enabled. Also shows current version of PreWare and Package Manager Service installed.
  do
 
    echo "Restoring $APP and unlinking..."
 
    unlinkapp
 
  done
 
}
 
# end of restoreall function
 
  
 +
'''Update PreWare'''
  
#-------------------------------------------------------------------------#
+
<code>mvapp upw<br>
# main - begins here
+
</code>
#-------------------------------------------------------------------------#
 
  
case $COMMAND in
+
Updates PreWare and Package Manager Services to most recent version.  You need to confirm install of each by answering 'y' for yes.
"clean")
 
  cleanapp
 
  ;;
 
"list")
 
  listapps
 
  ;;
 
"listmoved")
 
  listmoved
 
  ;;
 
"link")
 
  linkapp
 
  ;;
 
"unlink")
 
  unlinkapp
 
  ;;
 
"bulkmv")
 
  bulkmv
 
  ;;
 
"restoreall")
 
  restoreall
 
  ;;
 
*)
 
  usage 1
 
  ;;
 
esac
 
  
cleanexit $code
 
  
</pre>
+
'''Update Terminal App'''
  
----
+
<code>mvapp upt<br>
 
 
 
 
- Be sure to set execute permissions if you missed the step above...
 
<code>chmod 755 /usr/bin/mvapp</code>
 
 
 
== Precautions ==
 
 
 
So far, no one has reported an app that has issues due to linking.  These are best practices to avoid issues.
 
 
 
- Be selective about what you move.  You may not want to move apps that store important information to you such as password lockers, memo apps, EverNote, Agenda, etc. Games, web content apps and information viewing apps should be safer to move as there is no data at risk.
 
 
 
- Close apps before moving them.  Do not move an app while it is open and running.  I actually tested this and figured some apps would break but none have yet.  It is still not wise to move apps while they are open and running.
 
 
 
== Commands ==
 
 
 
'''To find the largest apps in /var/usr/palm/applications'''
 
 
 
<code>mvapp list</code>
 
 
 
It shows the largest apps last... IE....<br>
 
<code>
 
8352 com.apnews.webos<br>
 
8512 com.fandango.app.fandango<br>
 
8672 com.palm.app.musicplayerremix<br>
 
10304 com.shortcovers.palm.pre<br>
 
10432 com.fusioncreativestudios.deadman<br>
 
10656 com.ulocate.app.where<br>
 
 
</code>
 
</code>
  
 +
Updates the Terminal app to most recent version.  You should not run this within the Terminal app.  Use it when accessing Linux through a computer.
  
'''To move and link an app to /media'''
+
'''To force tar restore'''
  
<code>mvapp link domain.appname</code>
+
<code>mvapp ftr domman.appname</code>
  
Example: mvapp link com.ulocate.app.where
+
This is a hidden function not shown in usage. It is for those who used an older version of mvapp and had tar backups. If you need to restore an app from tar, use this command. Or just remove the app from Launcher and reinstall if necessary.
  
The app should now work in the new location thanks to the link. Test each app to make sure it works before doing another. Looping the script to do every app in the dir or to link the entire app dir is not recommended. File attributes are lost on the move, which some apps may need. You need to test each app after a move.
 
  
Continue moving apps until it reports that /var.../applications is about 25MB or less.
+
'''To remove tar backups'''
  
There is an option to turn on/off tar-based backups.  See the BACKUP variable in the script.  If set to 1, it will perform a tar backup.  This takes up more space on /media but if you need to restore, it puts it back with full file attributes.  Recommend keeping this on.
+
<code>mvapp rtb</code>
  
 
+
This is a hidden function not shown in usageTar backups have been disabled by default since mvapp 0.2.4Tar backup has been turned off because it wastes space on /media and has potential issues with restoring older versions of an app. Its purpose was to restore special file attributes not supported by FAT filesystem. The script no longer moves apps that FAT fs cannot support, so tar is no longer needed.  If you no longer need old tar backups, use this command to remove the entire backup directory.
'''To move and link many apps'''
 
 
 
<code>mvapp bulkmv</code>
 
 
 
This will show largest app first, ask if you want to move/link and then moves on to the next largest app.  If you answer no to an app, it will skip to the next.  Answer 'q' to quit.  Is easiest to use this method directly on a device using the Terminal app.
 
 
 
 
 
'''To revert the move and delete the link'''
 
 
 
<code>mvapp unlink domain.appname</code>
 
 
 
The application will be moved back to the original directory. File attributes are lost on the move if tar backup is turned off, which some apps may need. You need to test each app after a move.  If tar backup is turned on (default - see BACKUP variable in script), file attributes will be restored.   
 
 
 
 
 
'''To list apps that have been moved and linked...'''
 
 
 
<code>mvapp listmoved</code>
 
 
 
This shows a list of apps that have been moved and linked to /media.   
 
 
 
 
 
'''To restore all moved apps back to original location'''
 
 
 
<code>mvapp restoreall</code>
 
 
 
If backup was turned on, this will restore ALL applications that have been moved/linked, back to the original location in /var with original attributes (using tar). If backup was not turned on, it will still attempt to move the applications back to /var using the files in /media, but without original file attributes. This may take several minutes to completeIt will show each application progress.
 
 
 
 
 
'''To cleanup and remove directories and symlinks'''
 
 
 
<code>mvapp clean domain.appname</code>
 
 
 
If you have any issues, first attempt to remove the app from the App Catalog or Homebrew app installer.  Then issue the 'clean' command to remove the applications directories at both locations and also the symlink.  You can then reinstall the app from the App Catalog or any Homebrew App installer.
 
  
 
== Contingency Plan==
 
== Contingency Plan==
Line 623: Line 214:
  
 
If the application still does now work correctly...<br>
 
If the application still does now work correctly...<br>
-- Remove the App using official methods<br>
+
-- Remove the App using official methods (remove from Launcher or Homebrew installer)<br>
 
-- mvapp clean domain.appname<br>
 
-- mvapp clean domain.appname<br>
 
-- Reinstall through the App Catalog or Homebrew installer app<br>
 
-- Reinstall through the App Catalog or Homebrew installer app<br>
 +
 +
If you have issues with email, follow these steps...<br>
 +
-- mvapp unlinkemail
  
 
== Proposal to Dev Community ==
 
== Proposal to Dev Community ==
Line 646: Line 240:
  
 
'' Fair Dinkum method''<br>
 
'' Fair Dinkum method''<br>
Rod Whitby's FD method works tactically but doesn't resolve the issue of filling up /var space.  A webOS update can use 30% of /var alone and the next update could have issues if too much /var space is used.  The symlink method not only saves space on /var, it can reduce it significantly when many apps are moved.
+
Rod Whitby's FD method works up to a point but if /var usage is over 100MB, the FD method no longer works.  The symlink method not only saves space on /var, it can reduce it significantly when many apps and email are moved to media drive.
  
 
''Resizing /var''  <br>
 
''Resizing /var''  <br>
Line 658: Line 252:
  
 
''Creating a loopback filesystem to a virtual file located on /media/internal'' <br>
 
''Creating a loopback filesystem to a virtual file located on /media/internal'' <br>
This has been worked on [http://www.webos-internals.org/wiki/Tutorials_Linux_opt_on_loopback here] and still has potential.  Unfortunately it locks out USB mount and media sync altogether.  If a workaround can be found with low risk, this may be the most ideal solution.
+
This has been worked on [http://www.webos-internals.org/wiki/Tutorials_Linux_opt_on_loopback here] and still has potential.  Unfortunately it locks out USB mount and media sync altogether.  If a workaround can be found with low risk, this may be the most ideal solution. 
 +
 
 +
Suggestion to Palm... consider virtually mounting /media so that both the kernel and USB can use /media simultaneously.  This would allow the loopback fs to work and would probably also allow phone calls while USB mounted.  This could become the strategic solution.
  
 
== Risks, Issues, Dependencies ==
 
== Risks, Issues, Dependencies ==
Line 664: Line 260:
 
- Some file attributes of linux fs are not possible on FAT fs (USB drive).  The 0.2.0 version (and higher) now deals with any apps that cannot copy attributes/permissions properly.  The script will not move apps that have special attributes or conditions that FAT fs can't deal with.  In general, this has been resolved.  No known issues with apps since version 0.2.0 or later.
 
- Some file attributes of linux fs are not possible on FAT fs (USB drive).  The 0.2.0 version (and higher) now deals with any apps that cannot copy attributes/permissions properly.  The script will not move apps that have special attributes or conditions that FAT fs can't deal with.  In general, this has been resolved.  No known issues with apps since version 0.2.0 or later.
  
- Some apps may not behave well if USB drive is mounted to computer, though I've tested several that behaved fine. Linux type background services probably would not work well so probably should not be moved, however I'm not aware of any service stored in the app home directory - they are usually setup somewhere else.  Apps that do IO to the home directory of the app while USB mounted may have issues when located on /media.  I'm not aware of any apps that write to their own home directory, so this may not be a risk in general, but is still a possibility.  If you run into issues with an app, see the Contingency Plan.
+
- Some apps may not behave well if USB drive is mounted to computer, though I've tested several that behaved fine. Linux type background services probably would not work well so probably should not be moved.  Apps that do IO to the home directory of the app while USB mounted may have issues when located on /media.  I'm not aware of any apps that write to their own home directory, so this may not be a risk in general, but is still a possibility.  If you run into issues with an app, see the Contingency Plan.
 
 
== Future Versions ==
 
  
Plans for future versions...
 
 
- The script will soon be modified to be friendly for javascript calls... no prompting and no output, just result codes.  Will still maintain user interaction capability as well.
 
  
 
== Confirmed Apps ==
 
== Confirmed Apps ==
Line 681: Line 272:
 
http://forums.precentral.net/web-os-development/205649-resolution-app-catalog-install-limit-proposal.html
 
http://forums.precentral.net/web-os-development/205649-resolution-app-catalog-install-limit-proposal.html
  
'''Your Experiences'''
+
== Source ==
  
Please post your experiences, good or bad. I'd like to get any kinks worked out before attempting to turn this into a webOS app.
+
The source has been moved to gitorious...
  
- How many apps did you move? <br>
+
http://gitorious.org/webos-internals/mvapp
- Did you find an app that won't work linked? <br>
 
- Did you move any back?  (please test)<br>
 
- How far down did you have to get /var.../applications down in MB before you could start adding apps from the App Catalog?<br>
 
  
 
== Credits ==
 
== Credits ==
xorg - initially developed script and proposal.  maintainer of this page.<br>
+
<pre>
daventx - added unlink and clean functions.<br>
+
xorg - initially developed script and proposal.  maintainer of this page.
 +
daventx - added unlink and clean functions, other suggestions<br>
 
nt4cats - found Fair Dinkum breaks mvapp, gave solution to use explicit path.
 
nt4cats - found Fair Dinkum breaks mvapp, gave solution to use explicit path.
 +
SirWill, hparsons, bclancy, navinag, dhcalva, chodaboy - testing, usable suggestions and feedback
 +
emoney22 - pointed out hidden directory in /media hides images from Photo app
 +
greg_roll - provided info to capture running apps
 +
rwhitby - setup webosinternals repository for homebrew distro
 +
 +
And thanks to the many testers who gave feedback on the PreCentral thread.
 +
</pre>

Latest revision as of 22:15, 10 December 2009

Overview

The 'mvapp' method is a Linux command that allows moving your Pre apps and email to the (much larger) media drive by using symbolic links. This will save space on internal system storage (/var) and allow more App Catalog apps to be installed, up to media drive limit (gigabytes) rather than the 64MB limit.

This method involves creating a (hidden) directory /media/internal/.apps, moving selected applications to the newly created directory, and then creating a symbolic link in the /var/usr/palm/applications directory pointing to the new location. mvapp v0.3.0 and higher now also moves email and attachments to /media/internal/.data. This frees up the disk space from the relatively limited internal system volume (/var) to the much larger media drive (/media/internal). The 'mvapp' command (available on PreWare) will move the files and create the appropriate link.

Solution (symbolic link method)

Install mvapp:
Now on PreWare in the Linux Applications category. In PreWare Preferences, need to have 'show available types' turned on.

Quick Start Guide
- Launch PreWare
- In PreWare Preferences, need to have 'Show All Package Types' turned on
- Install 'mvapp' from PreWare in the Available Other -> Linux Application category
- Install Terminal app from PreWare (if not already installed)
- Open Terminal from Launcher, then type...

mvapp doctor
mvapp bulkmv

Prepping for webOS 1.3.5 Update

There's a pretty good chance the next webOS update will address the app limit storage issues, which will mean 'mvapp' will likely not be needed from that point on.

Best Case Scenario: There is the possibility that mvapp will be compatible with 1.3.5. It may be able to maintain the links with the new app storage method. If the case, you can do nothing or 'mvapp restoreall' after the update. I'll likely able to answer this before 1.3.5 is released.

Otherwise...

Pre Update Option: In preparation, just before upgrading to webOS 1.3.5, you can perform a 'mvapp restoreall' to restore apps and 'mvapp unlinkemail' to restore email back to the /var location.

However if you have accumulated dozens of apps and/or a large amount of email, you may not be able to restore fully and have enough room for the upgrade. You may need to do some cleanup by removing apps that you do not use. After the update, you should be able to reinstall your deleted apps. If willing, you may also consider deleting your email accounts and reinstalling before performing 'mvapp unlinkemail'. Though I suspect that linked email will be compatible with 1.3.5.

Post Update Option: I'm planning to have a secondary recovery option for those who perform an OS update before getting a chance to restore mvapp links, but it won't be released until the day of the update. Hopefully 1.3.5 will be compatible with existing mvapp links and no preparation will be needed.

Follow this section for more details.

Precautions

So far, no one has reported issues with apps or email due to linking. These are best practices to avoid issues.

- Version 0.3.0 and higher now can move email and attachments to media drive. It is unknown how a future webOS release will deal with the storage issues. It's possible a future webOS release could break the email link to the media drive. While I'd likely have a new mvapp release out before next webOS release, losing email on the phone is a possibility and you may have to resetup email accounts. Moving email is not recommended if you rely on email for your job unless you don't mind possibly losing all email on the phone. Your email will still exist at the original source (gmail, yahoo, hotmail, etc), so linking email is not likely to have much of an impact for most of us.

- If Palm releases a new webOS that resolves the overall issue, you _may_ (but not likely) need to completely restore your apps back to /var before performing the upgrade - mvapp restoreall. For email - mvapp unlinkemail. If a future webOS update is applied before you get a chance to restore, it's possible it could break this method, so moving apps that store important information to you is not recommended. Since it would likely be a significant architectural change for Palm to fix this issue, storage issues may not be officially resolved until webOS 2.0, but who knows. webOS 1.3.1 does not have any issues with mvapp.

- This is a workaround script not approved by Palm. The usual disclaimers apply.

Commands

To run diagnostics on /var and get recommendations

mvapp diag
(or shortcut)
m d
m d m (includes media apps, slower)

This reports the usage of applications, email and attachments on /var. It also recommends actions based on the results.


To find the largest apps in /var/usr/palm/applications

mvapp list
(or shortcut)
m ls

The first number is size of app in KB, largest shown last. IE....
8352 com.apnews.webos
8512 com.fandango.app.fandango
8672 com.palm.app.musicplayerremix
10304 com.shortcovers.palm.pre
10432 com.fusioncreativestudios.deadman
10656 com.ulocate.app.where


To move and link an app to /media

mvapp link domain.appname
(or shortcut)
m ln domain.appname

Example: mvapp link com.ulocate.app.where

The app should now work in the new location thanks to the link. Test each app to make sure it works before doing another.

Continue moving apps until it reports the 'var used' is below 100MB. If 'var used' is 95MB, you could add 5MB more apps.


To move and link many apps

mvapp bulkmv
(or shortcut)
m bm

This will show largest app first, ask if you want to move/link and then moves on to the next largest app. If you answer no to an app, it will skip to the next. Answer 'q' to quit. Is easiest to use this method directly on the phone using the Terminal app, available in Homebrew.

To search for apps by partial app name

mvapp search
(or shortcut)
m s

This is an easier way to link/unlink/clean apps from the Terminal app. You only need to supply a partial app package name. It will show any matches then ask what you'd like to do with it.

To restore the app to original location and delete the link

mvapp unlink domain.appname
(or shortcut)
m ul domain.appname

The application will be moved back to the original directory. If you have issues with an app after unlinking, see the Contingency Plan section below.


To list apps that have been moved and linked...

mvapp listmoved
(or shortcut)
m lm

This shows a list of apps that have been moved and linked to /media.


To restore all moved apps back to original location

mvapp restoreall
(or shortcut)
m ra

This will restore ALL applications that have been moved/linked, back to the original location in /var. This may take several minutes to complete. It will show each application progress. Beware that if you have moved a very large number of apps over a period of time, they may not all fit on /var. You may need to remove some/many unused apps before restoring all.


To cleanup and remove directories and symlinks

mvapp clean domain.appname
(or shortcut)
m c domain.appname

If you have any issues, first attempt to remove the app from Launcher, the App Catalog or Homebrew app installer. It's very important you remove from the phone first before using the clean command. Then issue the 'clean' command to remove the applications directories at both locations and also the symlink. You can then reinstall the app from the App Catalog or any Homebrew App installer.


To move and link email and attachments

mvapp linkemail
(or shortcut)
m le

This will move all of your email from /var to /media/internal/.data/emails. This will close the email application so if you are writing a new email, be sure to send the email before using this commend. Also see the Precautions section above.

To restore and unlink email and attachments

mvapp unlinkemail
(or shortcut)
m ue

This will restore all of your email from /media back to /var. This will close the email application so if you are writing a new email, be sure to send the email before using this commend.

Hidden Commands

These commands are not documented in the Usage display.

Check Links

mvapp checklinks
(or shortcut)
m cl

Checks if apps in /media have a link. If not, you have the option to remove the app from media, which is recommended. This can occur if the app was not fully removed when you removed from Launcher or homebrew installer.

Check if Fair Dinkum is installed

mvapp checkfd
(or shortcut)
m cfd

Checks if Fair Dinkum is/isn't installed and enabled. Also shows current version of PreWare and Package Manager Service installed.

Update PreWare

mvapp upw

Updates PreWare and Package Manager Services to most recent version. You need to confirm install of each by answering 'y' for yes.


Update Terminal App

mvapp upt

Updates the Terminal app to most recent version. You should not run this within the Terminal app. Use it when accessing Linux through a computer.

To force tar restore

mvapp ftr domman.appname

This is a hidden function not shown in usage. It is for those who used an older version of mvapp and had tar backups. If you need to restore an app from tar, use this command. Or just remove the app from Launcher and reinstall if necessary.


To remove tar backups

mvapp rtb

This is a hidden function not shown in usage. Tar backups have been disabled by default since mvapp 0.2.4. Tar backup has been turned off because it wastes space on /media and has potential issues with restoring older versions of an app. Its purpose was to restore special file attributes not supported by FAT filesystem. The script no longer moves apps that FAT fs cannot support, so tar is no longer needed. If you no longer need old tar backups, use this command to remove the entire backup directory.

Contingency Plan

If you have problems with an application, follow these steps...
-- Close the application if open
-- mvapp unlink domain.appname
-- Try using the App

If the application still does now work correctly...
-- Remove the App using official methods (remove from Launcher or Homebrew installer)
-- mvapp clean domain.appname
-- Reinstall through the App Catalog or Homebrew installer app

If you have issues with email, follow these steps...
-- mvapp unlinkemail

Proposal to Dev Community

This is a formal proposal to the Dev Community suggesting that PreWare, WebOS Quick Install and other Pre installer apps provide an option to move any app in /var to the /media fs and create a link similar to the code above.

The Homebrew Community somewhat created part of the storage problem so needs to come up with their own solution. The symbolic link proponents propose that Homebrew apps be moved with a link to /media/internal by default and physically use /var only if needed (per conditions stated below). The developer would put a flag in the package (or some other method during submission) to state their app is able to run linked to /media or if it specifically needs to physically be on /var. Will propose additions to Packaging Standards to support /media links. The homebrew installer apps could then automatically do the move/link if the package is flagged for it.

Candidate apps for moving to /media
- apps that do not depend on file attributes not supported by FAT (such as sticky bit or internal links) - The script (post 0.2.0 now deals with this).
- apps that do not perform data operations to home app directory when device is USB mounted

Exceptions for maintaining apps on /var
- apps that depend on file attributes not supported by FAT - The script (post 0.2.0 now deals with this).
- apps that won't work well when device is USB mounted, such as performing data or DB operations in home app directory

(Please update with other known candidates/exceptions)

Benefits over other methods

Fair Dinkum method
Rod Whitby's FD method works up to a point but if /var usage is over 100MB, the FD method no longer works. The symlink method not only saves space on /var, it can reduce it significantly when many apps and email are moved to media drive.

Resizing /var
One challenge with resizing /var is that it will still have a fixed static limit - how do you decide how much to increase it? Many will still probably hit the limit or waste space if setting too high. There is also the warning from Palm that resizing var may interfere with future updates. The link method allows to dynamically use the /media partition, so there is no need to set a specific size dedicated to apps. If the USB drive is filled, users can decide if they use the space for media or apps on the fly.

AppPath in /etc/luna.conf
I proposed a while back adding an AppPath to luna.conf to include apps stored on /media. Some apps would not work because some apparently reference /var. IE, the vampire/mafia/quest series could not access graphics. The symbolic link fixed this because apps think they are in /var.

Link/mount all of /var/usr/palm/applications/ to /media/internal
If some apps rely on file attributes this won't work since file attributes are lost when moved to FAT fs. While this is rare, it probably isn't wise to force all /var apps to /media. Selectively moving apps one at a time is less risky. Update: Have found that "PDF View" app does not allow to be linked, so it appears that selectively moving apps is necessary.

Creating a loopback filesystem to a virtual file located on /media/internal
This has been worked on here and still has potential. Unfortunately it locks out USB mount and media sync altogether. If a workaround can be found with low risk, this may be the most ideal solution.

Suggestion to Palm... consider virtually mounting /media so that both the kernel and USB can use /media simultaneously. This would allow the loopback fs to work and would probably also allow phone calls while USB mounted. This could become the strategic solution.

Risks, Issues, Dependencies

- Some file attributes of linux fs are not possible on FAT fs (USB drive). The 0.2.0 version (and higher) now deals with any apps that cannot copy attributes/permissions properly. The script will not move apps that have special attributes or conditions that FAT fs can't deal with. In general, this has been resolved. No known issues with apps since version 0.2.0 or later.

- Some apps may not behave well if USB drive is mounted to computer, though I've tested several that behaved fine. Linux type background services probably would not work well so probably should not be moved. Apps that do IO to the home directory of the app while USB mounted may have issues when located on /media. I'm not aware of any apps that write to their own home directory, so this may not be a risk in general, but is still a possibility. If you run into issues with an app, see the Contingency Plan.


Confirmed Apps

Apps Confirmed Not to Work

com.palm.app.pdfviewer (script v0.2.0 and higher properly handles this, won't allow it to move)

Discuss

Discuss in the Discussion tab or PreCentral...

http://forums.precentral.net/web-os-development/205649-resolution-app-catalog-install-limit-proposal.html

Source

The source has been moved to gitorious...

http://gitorious.org/webos-internals/mvapp

Credits

xorg - initially developed script and proposal.  maintainer of this page.
daventx - added unlink and clean functions, other suggestions<br>
nt4cats - found Fair Dinkum breaks mvapp, gave solution to use explicit path.
SirWill, hparsons, bclancy, navinag, dhcalva, chodaboy - testing, usable suggestions and feedback
emoney22 - pointed out hidden directory in /media hides images from Photo app
greg_roll - provided info to capture running apps
rwhitby - setup webosinternals repository for homebrew distro

And thanks to the many testers who gave feedback on the PreCentral thread.