United Cavers Exploration Team

Cave and Mine Exploring => Website Issues => Topic started by: Dave Tyson on Feb 24, 2025, 01:14 AM

Title: Forums all loaded up on the new test site
Post by: Dave Tyson on Feb 24, 2025, 01:14 AM
Dear All,

I have managed (with a lot of work!) to move all of the significant forums from this website onto the test server at https://ucet.anduin.org.uk. As noted before all the user details are there apart from the passwords so you need to use the 'forgotten password' link. All the posts seem to be OK, but I need to copy over the attachments. This will be another significant tranche of work as the images may need some processing.

Feel free to have a look around - but note that I cannot guarantee all posts will survive as changes may take place, but you are welcome to test.

I have removed an inordinate amount of spam posts from this forum before the data copy and the new forum is largely spam free. It needs 'prettyfication' and a few features like thank-you tags.

While processing the data I discovered a lot of inconsistencies and so the topic counts on the new forum are probably more accurate, however the post counts do need to be recalculated.

All the data is from a snapshot on 20th Feb. I really don't want to have do do a full download of the data again as there is a lot of work converting the UTF-8 characters like ¼ etc. as the mysql output format encodes differently to postgres. I think I may be able to get a incremental snapshot when the time comes for a switchover.

As usual let me know if you find problems or have comments dtyson@anduin.org.uk

Cheers,
Dave
Title: Forums all loaded up on the new test site
Post by: Dave Tyson on Feb 24, 2025, 05:30 PM
Right, I have found one issue. It appears some posts are missing carriage return.line feeds and so are rather compressed. I'll have to investigate the cause, but suspect it's to do with having to deal with the line feed breaking messages up. I may need to reload the messages again as I think the 'squish' program will have to convert these characters so something different and the 'procm' program will have to do the reverse. Ho hum - like British Rail we are getting there - but slowly...

Dave
Title: Forums all loaded up on the new test site
Post by: Dave Tyson on Feb 25, 2025, 11:40 PM
I have managed to fix the problems with the lack of newlines in posts and so as far as I can tell everything is OK and the text content of posts matches that on the Joomla/Kunena forum. I have checked that the UTF-8 characters like £ â,¬ etc are OK. The data is all from the snapshot of 20th Feb 2025.

Critical things which need to be done now are:

1) load all the attachments (documents and pictures etc.)
2) add a thank-you system
3) regenerate the post indexes to match reality

Important things to look at soon:

1) Sort out moderators and administrators
2) choose a front end package to prettify the site
3) sort out what we are going to do about hosting and a date for switchover.
4) have a serious think about the knowledge base
5) maybe have a clear out of old and out-of-date info

Above all it needs users to actually login to the test site (so their encrypted passwords are cached - this will save pain later) and try things out. Have a look at personal settings and play with the personal message system. Try posting  messages (probably best in 'website issues' to avoid polluting other boards)

I would prefer to be able to add any changes which happen on the production system as an incremental update to save reprocessing all the data. If push come to shove I can do this as the programs are in place to reformat and load the data - however processing attachments may be very time consuming.

There is a good guide to the forum software (from the point of view of users and administrators at:

 https://wiki.simplemachines.org/smf/Main_Page

It gives a lot of useful info. Please report any problems you find to me as the earlier I know the sooner I can fix them :-)

Cheers,
Dave
Title: Forums all loaded up on the new test site
Post by: Dave Tyson on Mar 07, 2025, 04:05 PM
Just a quick heads up about the test website. I have loaded all the attachments apart from about 3 which are too big for the current forum limit. AFAIK everything should work OK. The forum software automatically makes a thumbnail of an image when it is first viewed - subsequent accesses will see the thumbnail - clicking on this brings up the full image.

Please report any issues to me.

Cheers,
Dave