X Z Verdana AfbY ( 08SLAND mac dataFileCleaner Prefs
dataFileCleaner Preferences
Cleaning of poorly encoded text from weblog posts and weblog titles will occur at the time the post is published. Event logging of any changes will always occur regardless of the setting that follows.
If you wish to see when dataFileCleaner processing happens click the following preference and look in your Events Log.
<%local(adrPrefs = dataFileCleanerSuite.utils.getPrefs());radio.prefs.checkboxPref ("Event Logging", @adrPrefs^.flLog)%>
Clean malformed entries in aggregatorData.root.
<%local(adrPrefs = dataFileCleanerSuite.utils.getPrefs());radio.prefs.checkboxPref ("Clean Aggregator", @adrPrefs^.flAggregator)%>
Clean malformed entries in weblogsComData.root.
<%local(adrPrefs = dataFileCleanerSuite.utils.getPrefs());radio.prefs.checkboxPref ("Clean Weblogs.com", @adrPrefs^.flWeblogsCom)%>
Clean malformed entries in weblogData.root.
<%local(adrPrefs = dataFileCleanerSuite.utils.getPrefs());radio.prefs.checkboxPref ("Clean Aggregator", @adrPrefs^.flWeblogPosts)%>
You can't use the above link to get anywhere that you expect to go. Instead Click Here.
Verdana % Ib\O = `;LAND mac #title "dataFileCleaner Docs"
#renderOutlineWith "fatHeadlines"
Version: <%dataFileCleanerInfo.version%>
dataFileCleaner Prefs
Documentation by Greg Hanek.
What does this Tool do?
The dataFileCleaner Tool does what its name implies. At the top of the hour, it will scrub (delete, actually) bad entries from the following root files used by Radio:
aggregatorData.root, where incoming RSS feed news stories are kept.
weblogsComData.root, where information about what weblogs have updated recently and pinged the weblogs.com server is kept.
weblogData.root, where your own weblog posts are kept.
These root files are located in the Data Files subdirectory of your Radio UserLand directory. Bad entries in these root files can affect certain operations of Radio and the display of data in your desktop Radio pages in your browser.
Every time this Tool deletes any bad items, it will also create an entry in your Events Log page explaining what action occurred.
How do I use it?
Menu choices in the Tools>DataFileCleaner submenu.
About: Brings up this page.
Preferences...: Brings up the dataFileCleaner Preferences page. You can change the same settings here as in the menu.
Run now: This triggers an immediate scrubbing of the selected root files. The particular root files are selected by the following three menu items.
These may be removed from the menu later as they are redundant.
Clean Aggregator: A checkmark indicates bad entries will be removed from the aggregatorData.root file. Selecting this menu item will toggle the checkmark on or off.
Clean Weblogs.com: A checkmark indicates bad entries will be removed from the weblogsComData.root file. Selecting this menu item will toggle the checkmark on or off.
Clean Weblog Posts: A checkmark indicates bad entries will be removed or fixed from the weblogData.root file. Selecting this menu item will toggle the checkmark on or off.
Event Logging: A checkmark indicates routine processes will be logged. Selecting this menu item will toggle the checkmark on or off.
Refresh Code...: This will update your copy of this Tool to the most recent version. Requires an Internet connection.
How do I know if I do need it?
Most Radio users will benefit from the use of this Tool, since it touches on the core functions that many users find important -- maintaining your own weblog, finding recently updated weblogs, using the news aggregator, and upstreaming.
The News Aggregator
Even if you delete all of the news stories you receive, some stories will still remain in your aggregatorData.root file. The news page will allow only the deletion of visible stories, but not all stories in your aggregatorData.root file will be visible. Some stories (those that contain only certain html code) will never be displayed on the news page, and thus cannot be easily deleted.
This Tool can remove such stories from your aggregatorData.root every hour. You won't see these files anyway, and if they are allowed to build up, they will only degrade the performance of Radio.
If you have a 'Macro error: Can't evaluate the expression because the name "channeltitle" hasn't been defined'.
Recently Updated Weblogs
Incomplete entries in the hourly update from Weblogs.Com will prevent the proper display of much of the data for the Weblogs.com On The Desktop (WOTD) page.
How can you tell if you have incomplete entries?
One possible indicator is the following message appearing in the status center of your Desktop Home Page (the page you post from):
Weblogs: Error reading Weblogs.Com data file.
If you see that, look in your Events Log page for a "Weblogs" Event entry that has a "What Happened" description like one of these:
Can't evaluate the expression because the name "ctChanges" hasn't been defined.
Can't evaluate the expression because the name "dateFirstAppeared" hasn't been defined.
Can't evaluate the expression because the name "dateLastChange" hasn't been defined.
Can't evaluate the expression because the name "url" hasn't been defined.
The presence of one of these messages confirms that at least one bad entry exists in your weblogsComData.root file.
Another clear indicator is the list of recently updated weblogs on your WOTD page is replaced by an error message.
Your Own Weblog
Under some circumstances (perhaps due to a crash or some other source of corruption), a completely empty weblog post (not just the text, but devoid of other post-related information as well) can exist in your weblogData.root file. The presence of such a "post" may prevent the proper display of your Desktop Home page.
How can you tell if you have incomplete entries?
You may see "Macro error:"
Can't evaluate the expression because the name "text" hasn't been defined.
Can't evaluate the expression because the name "when" hasn't been defined.
Under certain circumstances upstreaming would not occur because Radio would choke on certain high ASCII characters. I've added a conversion of the post text and title to more standard characters.
$datatabl h ++ $datatabl h ++ $datatabl h ++ $datatabl h ++ $datatabl h ++ $datatabl h ++ $datatabl h ++ $datatabl h ++ $datatabl h ++ $datatabl h ++ $datatabl h ++ $datatabl h ++ $datatabl h ++ $datatabl h ++ $datatabl h ++ $datatabl h ++ $datatabl h ++ $datatabl h ++ $datatabl h ++ $datatabl h ++ $datatabl h ++ $datatabl h ++ $datatabl h ++ $datatabl h ++ $datatabl h ++ $datatabl h ++ $datatabl h ++ $datatabl h ++ $datatabl h ++ $datatabl h ++ $datatabl h ++ $datatabl h ++ $datatabl h ++ O $ bR`bR
prefs XuserDocs VVerdana 2@8 n j 8 Eb*4
! flAggregatorflLog
flWeblogPosts flWeblogsCom n O K $ .]nv
docs prefs ! O FFFF000000060001001777AA000000000066000000001001710000001-7FFF6FBE18000004800480627200-10627222FFFF01031000000100000035000000010100840627222D5028C0001C627EE000020000010600000000 100,The dataFileCleaner Tool does what its name implies. At the top of the hour, it will scrub (delete, actually) bad entries from the following root files used by Radio: aggregatorData.root, weblogsComData.root, and weblogData.root. Co-Authored by Greg Hanek.00040000000A000000020102000050000000A0000000201020000600000031000000010920FFFF0900000
20 0007000000190000000102018000080000009A00000001 40,Verdana 40, 401C230 10000900000015000000023C21F03C21F0000A0000001500000002627222627222000B00000005000000020000C0000001800000001100E5262212E80000F0000005200000000400 10 4,77A 4, 4,CӀ 4,d1 4,X 4,ft 4,p 00130000007E00000000DA91F8CE1C1D1E1F7F1B0 44,- . . ' " FFFF0FFFF0FFFE000000060001001777AA EF
3x{ V 2]nv " 9
E Q ] ]nv author Andy Fragen authorEmail andy@thefragens.com description "z homePageUrl -http://radio.weblogs.com/0001017/publicTools/name dataFileCleaner releaseDateversion 0.82 ? Verdana .g J .LAND mac This script runs every hour.
local
fourHours = 60 * 60 * 4
if clock.timerExpired (string.popFileFromAddress(this), fourHours)
dataFileCleanerSuite.run (flHourly:true)
? 8 4 .r
everyHour )2 8 00 h Verdana 5Rf x LAND mac on publishItem (adrPost, flLogEvery = false)
Changes
03/23/05; 8:03:31 AM by AJF
Commented out stuff that alters adrPost^.text
03/19/05; 7:00:07 PM by AJF
Added stuff for new scripts
03/18/05; 3:49:19 PM by AJF
Created for callback
local
starticks = clock.ticks ()
ct = 0
flLog = false
postNum, preText, postText, preTitle, postTitle
adrPrefs = dataFileCleanerSuite.utils.getPrefs()
s
bundle //initialize intial values
table.assign(@adrPost^.text, string(adrPost^.text))
preText = adrPost^.text
if defined (adrPost^.title)
preTitle = adrPost^.title
postNum = string.popFileFromAddress (@adrPost^)
bundle //fix post text
if system.environment.isMac
table.assign(@adrPost^.text , string.iso8859encode(adrPost^.text, @html.data.iso8859.mac))
table.assign(@adrPost^.text, html.cleanForExport(adrPost^.text))
if system.environment.isWindows
table.assign(@adrPost^.text, string.iso8859encode(adrPost^.text, @html.data.iso8859.win))
bundle //fix post title
if defined (adrPost^.title)
bundle //run through string.iso8859encode
if system.environment.isMac
table.assign(@adrPost^.title, string.iso8859encode(adrPost^.title, @html.data.iso8859.mac))
table.assign(@adrPost^.title, html.cleanForExport(adrPost^.title))
if system.environment.isWindows
table.assign(@adrPost^.title, string.iso8859encode(adrPost^.title, @html.data.iso8859.win))
bundle // fix errant characters
table.assign (@adrPost^.title, string.replaceAll (adrPost^.title, "\"", """))
bundle //check for changes
postText = adrPost^.text
if defined (adrPost^.title)
postTitle = adrPost^.title
try //need to put in try block or split out check for title if title not defined
if !(preText equals postText) or !(preTitle equals postTitle)
flLog = true
ct++
if not dialog.yesNo (string.popFileFromAddress(@adrPost^) + cr + adrPost^.text)
break
wp.newTextObject (adrPost^.text, @adrPost^.text)
bundle //logging
if flLog
radio.log.add ( "dataFileCleaner", "Cleaned bad character entries from " + postNum + " title.", starticks)
if flLogEvery //testing
dataFileCleanerSuite.utils.log ( starticks, "Cleaned " + postNum )
bundle//test
new (tabletype, @system.temp.adrpost)
system.temp.adrpost.text = ("Viol")
system.temp.adrpost.title = ("This is a 15\" shark")
cleanPost (@system.temp.adrpost, "Posts")
edit(@system.temp.adrpost)
publishItem (@weblogData.posts.["00000438"], true)
: 6 0`
publishItem ` * : 09 n j . 0ڳ
'
,fileMenu *radio 6user 6 n x Verdana 5RfO[ [ iLAND mac on cleanPost (adrObject, flLogEvery = false)
Changes
03/22/05; 9:02:46 AM by AJF
Removed thread.callScript as it spawned too many threads
03/19/05; 6:48:02 PM by AJF
Made changes to incorporate new code
03/18/05; 4:16:05 PM by AJF
Changed to call callback cleanSinglePost to do the work.
local (i, sizeTable = sizeOf (adrObject^), starticks = clock.ticks())
local (adrPrefs = dataFileCleanerSuite.utils.getPrefs())
adrObject = string.popFileFromAddress (adrObject)
dataFileCleanerSuite.utils.log ( starticks, "Begin cleaning " + adrObject + " entries.")
adrObject = string.popFileFromAddress (adrObject)
for i = sizeTable downto 1
if adrObject == "weblogData.posts"
adrPost = @adrObject^[i]
dataFileCleanerSuite.callbacks.radio.publishItem (adrPost, flLogEvery)
dataFileCleanerSuite.utils.log ( starticks, "Completed cleaning " + adrObject + " entries.")
bundle //test
cleanPost (@weblogData.posts, true)
V Verdana Ӓ]n7 < LAND mac on cleanTable (adrObject)
Changes
03/22/05; 11:11:38 PM by AJF
Added continue after calling deleteTable or trashTable
03/19/05; 6:47:23 PM by AJF
Made changes to incorporate new code
local
starticks = clock.ticks ()
ct = 0
flLog = false
i
sizeTable = sizeOf (adrObject^)
adrObject = string.popFileFromAddress (adrObject)
dataFileCleanerSuite.utils.log ( starticks, "Begin cleaning " + adrObject + " tables.")
on deleteTable ()
flLog = true
ct++
delete (adrTable)
on trashTable ()
local (adrTrash = parentOf (parentOf (adrTable^)^) + ".trash")
flLog = true
ct++
table.move (adrTable, adrTrash)
if (sizeTable > 0) && (defined (adrObject^))
for i = sizeTable downto 1
local (adrTable = @adrObject^[i])
case adrObject
"weblogsComData.weblogs"
if sizeOf (adrTable^) < 4
deleteTable ()
continue
"aggregatorData.stories"
if sizeOf (searchEngine.stripMarkup (adrTable^.storytext)) == 0
trashTable ()
if not defined (adrTable^.channeltitle)
trashTable ()
continue
if not defined (adrTable^.storyText)
trashTable ()
continue
if (string.trimWhiteSpace (searchEngine.stripMarkup (adrTable^.storyText)) == "") and (not (string.lower (adrTable^.storyText) contains "![]()
a]LAND mac This script runs in its own thread.
loop //loop until the application quits
Add your code here.
local
toolName = "dataFileCleaner"
adrToolSuite = @[toolName + "Suite"]
adrToolThread = @[toolName + "Thread"]
local (starticks = clock.ticks())
adrToolSuite^.instal ()
adrToolSuite^.run ()
adrToolThread^.enabled = false
T P . 1]nv
ctenabledscript T t Arial
F F 2LAND mac on finalFilter (pta)
pta^.renderedText = string.replace (pta^.renderedText, "[[title]]", "")
return (true)
- Geneva Eu`Q 2LAND mac on firstFilter (adrPageTable)
return (true)
* [ Geneva Eu7a v L 3*lLAND mac on pageFilter (pta)
if typeOf (pta^.template) == outlineType or typeOf (pta^.template) == wpTextType
table.assign (@pta^.template, string (pta^.template))
if typeOf (pta^.template) == stringType and not pta^.indirectTemplate
pta^.template = string.replace (pta^.template, "<title>", "[[title]]")
html.addPageToGlossary (pta)
return (true)
y . EF
$ 0
; finalFilter p S firstFilter p ]
pagefilter p VGeneva P E $ K G 72 url !http://127.0.0.1/dataFileCleaner/ K O K $ linetext Homepath index.wsf O /
Home p Y VArial W space /GIFfGIF89a ! , D ; VArial V " + 0 : C L Q [ aalink 008000bgcolor FFFFFFflRenderlink 0000FFspaceGif spacetext 000000vlink 800080 VArial n T VArial n ; Verdana sbR iLAND mac return (renderObject (@dataFileCleanerData.docs.userDocs))
A Verdana 50bR pX8LAND mac return (renderObject (@dataFileCleanerSuite.utils.prefsBrowser))
\ ` 7bR
*
4 @
H T
[ g
n z
#filters ##ftpSite #glossary #images I#prefs #tools 1index prefs p VVerdana ??8 \ = L .\
)
= I
^ j
#enableddataFileCleanerData " dataFileCleanerInfo (dataFileCleanerSuite pdataFileCleanerThread dataFileCleanerWebsite ; V
Lucida Grande 6 h = Verdana .]no ' LAND mac #enabled
dataFileCleanerData
dataFileCleanerInfo
author
authorEmail
description
homePageUrl
name
releaseDate
version
dataFileCleanerSuite
background
everyHour
callbacks
fileMenu
radio
publishItem
user
cleanPost
cleanTable
codeFreshener
loadNewCode
refreshCode
refreshCodeMenu
releaseNewCode
dev
debug
debugPosts
oldScripts
instal
menu
run
statusCenterMessage
utils
dataFileCleanerThread
ct
enabled
script
dataFileCleanerWebsite
ev eu ev eu ev eu ev g ev g ev g ev g ev g ev g ev g ev eu ev evh ev evt ev evh ev ev ev ev ev ev ev ev ev evh ev evh ev evh ev ll ev ll ev ll ev ll ev evh ev ey< ev ey< ev ey< ev evh ev evh ev evh ev evh ev evh ev eu ev ev ev ev ev ev ev eu = @x
Lucida Grande
Lucida Grande g Geneva ^}
Lucida Grande mn
Lucida Grande
Lucida Grande