Software-based floppy disc data separator

Hi Phil, I didn't want to make a big deal of all this, but let me just publically answer some of the things you mention here. On 18 Jun 2010, at 02:42, Philip Pemberton wrote: Interesting, it seems the exact message you get is, "Ticket is closed, please create a new ticket". I forgot it did that. I guess we just thought that since people receive all the information in emails, there wasn't much point keeping it around (it was from 2007). If you lost those emails, surely you could just have created a new ticket to ask us to resend that information? Anyway, I guess this has been sorted out now. Lack of info in the WIPs? I'm not sure what you mean... Over the years we have posted over 1.5 Mb of text (in wiki markup) about what we have been doing, totalling nearly 90,000 words, and that is just the technical WIP reports - it doesn't include any news/knowledge base/glossary/article posts. I doubt very much many technical projects have documented their progress nearly as much as we have. We also went into a huge amount of detail about our format description language, and how it worked in this WIP: http://www.softpres.org/wip:2002-07-18. This includes explanations of all the core concepts, a format example, and a line-by-line breakdown explaining exactly what that example does. This seems to be the one you're talking about, so I am not sure how you missed that. I should say that our technology has changed a lot since 2002, but many of the more basic principles are explained there. Lack of info is not something I think we can be accused of. It's certainly not intended to be. Maybe we're perceived as such because we do not have a forum on our site? We didn't post it at that time mostly because of concern about laws against making reverse-engineering information about copy protection public. This is exactly why source code dealing in copy protection for Cedega/WineX, Daemon Tools, and similar tools have never been available - it would be illegal to do so. That was one of the reasons we put off making source available. We don't worry so much about this on such obsolete media these days, especially since exemptions were added to the DMCA thanks to work done by Stanford and the Internet Archive, among others. What you seem to want, and please correct me if I'm wrong here, is for us to go to a great deal of effort to document the painstaking research we have done over many years and thousands of hours, so you can have a look at it. I understand why you might be interested in that information, but virtually nobody asks for it (actually none that I can think of apart from you), and our priorities firmly lie with spending our time on the preservation aspects that affect everybody. Very likely we're already "too late" for many games as it is. No, it's currently just a concept in our heads. It's a simple thing though, we just haven't got round to implementing it yet. That was a minor reason, and only important when we were started producing IPF files. If you read the knowledge base article (http://softpres.org/faq:general:open_source) on the subject, you can see that it comes under "other reasons" right at the bottom of the page. Also, note the title of that page: "Will you release the file format and/or source code for handling IPF files in the future?" and the first line is: "Yes, absolutely. To not do would be rather flawed preservation." Yes, that is the generally the sort of thing we planned, except our public key would be part of the IPF library, and we would expose that information to emulators. It's not a huge amount of work, but it takes more time away from preservation activity, and so has been a low priority. Anyway, I've already said we're going to delay signing IPF files. It's something that can be done afterwards. Even better, maybe somebody will contribute that code. It probably wouldn't help you much anyway to be honest. Freeform had limits to what you can do with it, and a design based on it wouldn't be as flexibile as it could/should be. I have to ask... Do you realise the scale of the work you are proposing to implement? We spent *years* figuring out all the complexities involved in that properly, with many hours a day, virtually every day, spent on it. It's not exactly something you can do as a "hobby" project IMO. There also hundreds and hundreds of disk formats out there (and that's just for personal computers). Are you intending to reverse engineer them all? Fortunately, we are rather obsessive about such things. Maybe you do plan to do all that, but I just thought I'd mention it! :) Maybe you should consider making the dump format and mastering format separate? Or at the very least separate by file extension. It is bound to confuse people even more if they are in the same file. We tried to be clear amount it, and called them two very different things, and many people still got confused. Either we weren't very clear, or it's just hard to understand, perhaps both. I'm really not sure what you're getting at. How many IPF images do you think there are that are not exclusively for the Amiga? I just checked - there are only eight, and they were only given to contributors 10 days ago. So, what you're really asking is: Why are there no publically released emulators that support those 8 IPF files that were created 10 days ago? :)) We been working on multi-system support for a few years. KryoFlux was a diversion that took us away from continuing that work because we realised we needed it to continue. Many of the emulator authors we were talking to at the time basically said, "come back when you have something for me to play with", and so we did. Maybe you have far more time than we do. For us, time spent doing things like that is time away from preservation activities. Maybe you'd find it interesting to work Copylock out yourself? Not only would it be informative, you'd able to use an IPF (e.g. Batman) and see what the game code does under emulation... BTW, we only need to decrypt Copylock in order to check the integrity of that track. The IPF contains the original encrypted data and the copy protection intact. KryoFlux is rock solid - that was a core design constraint. Being in beta does not imply it is unstable, it just implies that we have not finished testing and that we make no guarantees. Our host software and firmware, as detailed in the WIPs, is almost entirely hardware independent, with only a very thin layer of hardware specific code in the firmware. It would of course, as you obviously realise, be entirely pointless to make a project like this and have it locked into a hardware platform that might become legacy in the future. We aimed to fix this problem for the long term. This is all detailed in the WIPs. Sounds good. KryoFlux does all of that now. I rather disagree with that. Unless classic old Bob has some very specific knowledge about magnetic recording, disk formats and copy protection, the best you can do is attempt a "best effort" auto-detection of what is on the disk. Anything outside of usual parameters (probably be okay if it's not a game), and you just can't give that with any real certainty. It might be "good enough" for many people, but I would strongly argue it is not good enough for preservation. We are planning to enable people to dump disks and write them back, but we need to make clear when they do so that there really are no guarantees. It's strictly done on a best-effort basis, which might be fine for standard disks (as long as you are unconcerned about authenticity). We will say that if people want a reliable disk, they would need to write one using a verified game from an IPF file - being a mastering format, it is exactly what it was designed for. That sounds good in principle, and I'm sure people will find it useful, but how do they know if an image file can be created or not? It might appear to work, but the purpose of many types of copy protection is to stay hidden, and it is an extremely non-trivial problem to detect such cases. That is what we worked on for years... and it's still not a entirely an automated process. We have built the tools to make it easier for a knowledgable person to do, and that was still a massive amount of work. Disks produced by a end-user driven method are more likely to be broken, and it may not be found out until the physical disks are dead. AFAIK, Catweasel works on XP (at least in 32-bit mode). Thanks, I appreciate that. I still detect some subtle negativity in this post, but perhaps it is to be expected if you see your product in some way in competition with ours. Do please notice how I haven't said anything about DiscFerret unless it has been either something positive, or some advice. Kieron