App Angles | wp7dev blog by Mick N

sharing experiences from windows phone development

• Home
• About

Thoughts on Marketplace App Spam

It’s not an uncommon sentiment, and there are likely to be different views around which I believe Microsoft will and should be listening to.

I’ve discussed the topic on Twitter and the AppHub and thought to make a more coherent case in this article. Please feel free to add to the discussion whether you agree or disagree, in part or whole.

The Story so Far

So far the story since Mix10 has been that the platform will have great search capabilities and the apps that people are looking for will bubble to the surface. The platform seems to be evolving with better search capabilities and if you listen to Windows Phone Dev Podcast » Episode 015 – “Winning With Watson” Brandon makes some good points in favor of keeping the submission process open. Two in particular stood out to me.. listen for comments on AltaVista and what else devs produce after their flashlight app gets their feet wet.

Priorities

Besides thinking that market forces will make the marketplace self-regulating to an extent, the first thing that comes to mind for me when hearing different points of view on this topic is a question.

What do the other two primary competing platforms do to address this? So far the advice I’m given is little/nothing and in some cases it’s worse.

This reinforces the view I’ve held so far which is that I believe quality (over quantity) should become more of a priority when app count is less of a distinguishing feature between platforms (with Apple currently boasting app count in TV advertising)… till then, I’d like to see search capability continue to be improved to help direct people to what they want and let Microsoft keep fighting the good fight on making the platform relevant in consumers eyes. Sad but true: most people don’t have time for details and it’s going to be the macro story that gets most people attention initially. Many of us will want to give Microsoft every chance at strengthening it’s hand at this stage.

I also believe there will come a time in the not too distant future that app count will be nailed and certification teams can start locking down on the value added by apps, particularly in certain keywords/categories where some devs may be submitting poor quality apps or stretched the friendship on current policies by excessively fragmenting apps.

Perhaps there would even be a case later for Microsoft making a campaign of driving marketplace quality up and app count down, turning the tables from being follower on what was historically the main marketplace metric to leader, redefining consumer expectations.

Could a Compromise Address Both Priorities?

While discussing it today on twitter a compromise for now vs later started to formulate … a strategy, if you will, for improving search capability now for consumers and preparing to improve on quality later through policy.

Statistics

Statistics could be collected now to factor in attributes such as

- “install followed by (near) immediate uninstall” ratio (a strong indicator for trash apps or poor expectation management in app descriptions)

- “time in market” to “install count” ratio (a strong indicator of excessively fragmented or improperly targeted apps when found in heavily populated keywords/categories)

- ratings

Let devs see the details of these statistics on their account, and how it (subject to tuning) might currently be massaged into a “Value Indicator” of sorts.

By looking at various factors such as these, it would be possible to filter out situational considerations that might lead to some apps being false positived for not having a particular statistic being usable or not (e.g. no ratings). A “weak indicator” if you like.

Search

When filtering and sorting in marketplace views allow this Value Indicator (or equivelant) to be factored in by consumers. Watch how consumers use it and tune the process.

Cleanout

Apps could also be flagged for removal from the app store if presenting poor value, particularly in heavily populated sub categories. I think we’d want Microsoft to be open about their thinking in this area before acting, taking feedback and where necessary giving devs the opportunity to review their approach before action is taken.

Scrutiny

If it were necessary, dev accounts could be flagged for closer attention if historically maintaining very poor value statistics – serving as a reminder if there were a case for anyone needing to be mindful of their approach.

Conclusion

In short if something along these lines were adopted, besides offering useful indicators to devs for self regulation, app count continues to grow, consumers would have improved searching capability to filter out what they consider to be noise and when marketplace app count is sufficient, stats can be used to drive quality over quantity in overpopulated keywords/sub categories.

In a way I think it follows the cycles of many strong businesses… there are growth stages where things are kept simple and allowed to grow organically, then the accountants and policy makers get brought in to bring back order and consolidate before the next growth stage.

Thoughts?

Inside the Mango Developer Tools (App Platform, Communications, Search) – 3 quick channel9 sessions

You might have seen these referred to in @ToddBrix ‘s recent blog post on Windows Phone Marketplace on the Web for Mango

Here’s the links for your convenience.

Inside the Mango Developer Tools: App Platform | Windows Phone 7 | Channel 9 with @BrandonWatson and @AndrewClinick

Inside the Mango Developer Tools: Communications | Windows Phone 7 | Channel 9 with @BrandonWatson and Ian Todd

Inside the Mango Developer Tools: Search | Windows Phone 7 | Channel 9 with @BrandonWatson and Jared Brown

@MickNDev

Mango to Support Access to Hidden WiFi networks

I was late to the live broadcast of the Tech Ed 2011 keynote but did manage to catch this gem from an article tweeted by @MaryJoFoley Microsoft to add new business features to ‘Mango’ Windows Phone update | ZDNet while I was tinkering away on seeing what NoDo has done to IsolatedStorageFile methods (see my previous article).

I first asked about Secure Wireless Access Point settings (SSID Broadcasting) and Reset hidden cached WAP settings on the AppHub Features and Suggestions Forum when they launched and followed up again on it when NoDo release notes hinted at improvements in the area.

It looks like we’ll have it in Mango though, so that’s awesome. I don’t like broadcasting SSIDs and I’m not alone there.

This is a big one, thanks again for listening guys!

NoDo Caching Observations for Windows Phone 7 Isolated Storage File Handles and an Update on Performance for Folders with Lots of Files (build 7390)

Earlier this year I reported on the performance impact of storing lots of files in isolated storage folders in the RTM build of Windows Phone (7004).

How many files are too many files for Windows Phone 7 Isolated Storage folders (performance)?

@JeffWilcox noted just prior to Q&A (50:45) in Analyzing and Improving Windows Phone Application Performance | MIX11 that performance considerations relating to folders with lots of files “probably should be made a little bit more known“… well hehe… trying to get that word out here… and Dave @SilverlightCream Campbell kindly picked the first article in the series up… if I make a really horrible title perhaps I can improve this article’s SEO for inquiring minds too

@ClintRutkas, @keyboardP and @ChrisWalshie kindly volunteered shortly following the initial article being published to execute the benchmark on production devices to verify the behaviour reported was not isolated to the prototype Samsung Taylor the original benchmarks were performed on.

Since then we’ve seen the February update (7008), the March [NoDo] update (7390) and an update related to fraudulent certificates (7392). Details on each build here.

The 7390 release brings a substantial improvement in this behaviour for certain access patterns in addition to the headlining performance optimisations.

To demonstrate, refer to the following chart.

These results, this time from an HTC Mozart, initially appeared to be an optimisation to IsolatedStorageFile.Open() file for the majority of invocations. It then occured to me I may be looking at the result of a change in the caching algorithm. Further tests revealed this appearing to be true.

In preparing the results for the initial article in this series I soon found performing Openfile on a file recently created with CreateFile() resulted in performance indicating a cache was in play. Ordinarily a good feature, it was in the way for the point of the exercise. I then implemented a fixed size queue. When the queue gets full, files fall out the back of it… effectively giving me a “lookback” n number of files mechanism where n is the size of the queue, with the downside of invalidating the first n OpenFile() results and (in the first articles case very slightly) staggering the chart for OpenFile() operations behind that of CreateFile(). I settled on 50 as the magic number for the first article.

This was far from adequate for test runs under NoDo.

My initial reaction was to choose a lookback of 2500 which turned out to be a fairly good call. I then ran a lookback of 650, 1850 and finally 3350. Bear in mind it would be unlikely for a real app to work this way. This is only aimed at simulating random access of files in a file system after the folder has been populated. Again, the downside of this lookback mechanism is invalidating the first n results.

Interestingly, there is a creeping linear degradation affecting both OpenFile() and CreateFile() which corrects itself over time (see after each OpenFile() spike). I haven’t explored this in detail, however do note that in my original data sent to the dev team there were other observations of this nature not necessarily related to Isolated Storage performance.

I’ll include charts for all 4 additional trials run (2500, 650, 1850, 3350), as the results were all interesting. I’ll then carry on with the 3350 lookback for discussion purposes which I let run through to completion as it was adequately illustrating that the issue reported in the initial article still exists, but is limited to the extent the caching mechanism is in effect, depending on the file access pattern of your app.

Backing out of the app and resuming the run demonstrated the cache mechanism withstanded the app’s lifecycle. As yet I haven’t tested restarting the phone when the app is backed out… it’s getting a little late for that tonight and I’d like to get this out now.

The failure at 10,837 files has not changed when benchmarked for files 1k in size.

This apparent change in cache handling will be a bigger win for apps that reopen files among the last 3000 or so created.

We can take another look at this when Mango publicly ships to see how the story is unfolding.

I’m expecting the simple declarative abstraction I’ve implemented to hide this performance problem from apps needing to store lots of files still works fine. I’ll run that test overnight and report back if there is anything of consequence to note.

I’m still happy to sanitize and open source the declarative abstraction that solves this problem for anyone that this can save some time for. The solution is theoretically unconstrained in how many files can be created without notable performance degradation. Thus far tested to ~250,000 files. The license will be permissive. I’ve temporarily pushed this down the priority list as the app it’s implemented for has had a priority bump in favor of another commercial project.

@MickNDev

UPDATE 16/5/11: The performance accomplished under 7004 at the end of the first article using the declarative abstraction (as previously configured) has not worked out the same under NoDo. This may be related to cumulative effects of the creeping performance degradation in the first chart in this post. I’ll begin by doing some trials with different configurations of the declarative abstraction and see how that looks. I’m also curious to drill down into where this cost is having a cumulative effect.

UPDATE 22/5/11: An elegant solution has been found to circumvent what turns out to be an apparently additonal and new performance penalty associated with repeat IsolatedStorageFile operations. I’ll post the charts and details in a follow up article.

Observations have been posted on twitter in the interim. Feel free to hit me up for any details in the meantime there also.

Congrats to the Windows Phone Team (listening to devs)

Word has travelled quickly regarding the great announcements made during Keynote 2 at Mix11 for the future of Windows Phone Development with the upcoming Mango release (If you haven’t seen it yet, check it out on the channel 9 site – more on this soon).

I’m as impressed as the next guy with all the great features announced for release with Mango, which I’ve no doubt has and will be commented on extensively in the press and blogosphere.

What I’d like to comment on here in this short post is what seems to me to be a fantastic job the Windows Phone Team and Microsoft has done in not just listening to the developer community’s feedback, but taking this feedback on board and delivering on it.

This is evident not just in the big ticket items announced during the keynote, but also in many of the details announced and demonstrated in the breakout sessions (also on the channel 9 site). More on this soon.

In the time I’ve spent since Mix10 fielding developer questions on Microsoft’s and Stack Overflow’s phone developer forums I’ve seen no shortage of “observations” about this feature or that feature being insufficient or missing for particular developer’s needs and I’ve read them all.

It was apparent early on that the team was listening to all this great feedback and in an effort to support this, I’ve written countless replies to these posts, over the past year, reassuring developers that it’s worth the effort to write their feedback constructively with a reminder that whilst the team does not offer commitments to development plans as issues are raised, they are actively monitoring the forums (in particular the new Feature Suggestions and Feedback Forum launched in October 2010) and incorporating developers feedback into the products development cycle.

I’m so pleased as I go through feature after feature, detailed at Mix11, to see so many of these are exact reflections of the issues people have been raising over the year that has passed. Even the issues that were only raised a few times.  This is clearly a testiment to the internal processes the team must be using to filter this information through to the development cycle. I’m pleased not just because this is great for the platform, but for the reassurance that my faith and public support in the product and team has been well placed.

So in short, to the guys and gals on the Windows Phone Team, and everyone at Microsoft who is supporting the platform’s development, Congrats! You’ve nailed it.

It’s only just begun, and there’s lots more to do. But if this is an indication of what to expect as the platform develops in the years to come I can honestly say I’m very happy to have invested in this platform.

Also, announced in Andrew Clinick’s DVC14 breakout session, a site and app dedicated to Customer Feedback for Windows Phone (refer around 50 minutes in).

@MickNDev

How many files are too many files for Windows Phone 7 Isolated Storage folders (performance)?

You may have wondered, like me, how many files does it take to degrade the performance of isolated storage for Windows Phone 7 apps. You may develop apps or libraries where this will matter or you may just be curious to accumulate knowledge that could be helpful later.

I began benchmarking this some time ago and along with some interesting findings on the topic of this article, there was a number of other interesting patterns emerging from profiling the supporting code. I shared this result with the platform team at the time. They seemed quite interested in the results and asked if I minded having it circulated internally to the perf guys.

The short answer to the topic for this article is it’s possible to structure your isolated storage file system in such a way where performance virtually does not degrade at all.

The long story is a bit more interesting. If apps or libs you develop create a lot of files without regard for performance, apps could be experiencing performance degradation which, depending on the usage, could vary from minor to very substantial.

What I will share with you here are observations from a substantial volume of benchmarks I’ve run to improve visibility of the impact that our design choices have on the performance of specific isolated storage methods. This will help us to make better choices.

What I will also share, hopefully shortly, in another article is a helper class that will allow you to make these decisions declaratively using a light weight abstraction that will be demonstrated to allow a virtually limitless number of files to be created with virtually no impact on performance and require no coding on your part. Sounds alright? Cool, lets take a look at what’s going on…

What methods are affected?

• IsolatedStorageFile.CreateFile()
• IsolatedStorageFile.OpenFile()
• IsolatedStorageFile.GetFileNames()

What is the execution time for these operations under optimal conditions?

CreateFile() and OpenFile() start out at around 10 milliseconds and under optimal conditions can be shown to have slow and steady, linear growth. Unfortunately, the optimal conditions shown in this chart cannot be relied on without intervention.

All charts in this article show

X Axis: File Count

Y Axis: Milliseconds

GetFileNames() execution time starts out at around 5 milliseconds and grows linearly, consistently.

How long can it take to do these operations if performance is not considered?

Performance tends to degrade in steps, apparently as resources are consumed from more efficient pools, giving way to using progressively less efficient allocation mechanisms. How long it takes for execution time to step up in this manner has been the subject of interest for many benchmarking runs under different conditions.

So here we’ve gone from around 10 milliseconds to approaching 300 milliseconds.

After 3,000 files (in this run) an x25 fold increase can be observed in the execution time to create 1 file. Combined with other operations your code performs or if writing a few files at one time this is already starting to add up at the business end – the user’s experience.

You will notice at times execution cost will remain on the slow and steady incline for some time, other times only a limited number of files can be created before the next step up in execution time. Other times, often later in the cycle, it will create very few files at each step seemingly increasing in execution cost much more rapidly.

In another sampling run these operations are seen taking 1 second, per file. 4,500 files in this case to see a x100 fold increase in execution time.

Why would I want my app to make lots of files?

Firstly because isolated storage operations take a non trivial amount of time to execute, particularly on large files.

If you are rewriting these files or reading in whole files when only a subsection of the file is required by the user, you can easily be motivated to design your storage system to make use of files on a more granular basis for substantial performance gains.

Home grown databases or even database libraries that allow generation of an arbitrary volume of data may opt to store files on a highly granular basis to reduce the memory footprint of apps, which for 256MB Windows Phone 7 devices is limited to 90MB.

An app could also be generating files in response to user actions that can be repeated an arbitrary number of times. Taking a photo, for example.

What makes a difference to the perf curve?

You may not be too surprised to read, the number of files you store per directory affects performance.

There appears to be an element of chance involved with just how many files must be created in a directory before the first step up in execution cost takes place.

The chance of this happening appears to be influenced to a degree by the naming convention you use for your files.

Performance degradation of GetFileNames() is consistently linear and is effected by nothing other than the number of files physically in the directory. GetFileNames() does not currently observe the searchPattern provided. For every 2500 files you’re looking at close to 1 second execution time.

Long enough that you probably would rethink why you are using this operation with that many files in a directory.

How many files can I create before performance degrades?

After a substantial volume of benchmarking on files of a variety of file sizes (1, 5, 50, 500KB, 1.5, 5MB) it would appear that after creating 128 files in any directory there begins to be a chance that CreateFile() and OpenFile() performance for that file will degrade beyond the minor linear degradation that takes place up to at least 128.

When using GUIDs for filenames the chance of this degradation happening on or soon after 128 files is fairly high as shown here.

Using GUIDs for generated files can be convenient because you get (statistically likely) unique filenames without having to maintain additional state, and without having to consider strategies for dealing with potential power disruptions during transactions that require atomicity.

If you use an 8 digit, zero padded, numerically incrementing sequence for filenames, the chance of performance degradation stepping up appears to be significantly lower into the high hundreds, however there is still a chance for it to kick in shortly after 128 files as can be seen here.

In this chart (above), at each 2000 file boundary the process is restarted. In the second batch, starting at 2000 on the X Axis, the first step up occurs early. In another, starting at 8000 on the X Axis you can see even after 2000 files are created there still has been no substantial jump in execution time.

In a long running benchmark aimed at finding the lowest file count at which this step up in execution cost can occur, it appears in trials done so far, that this step up does not occur before 128 files. In fact over 760 tests of 500 files each, besides those that did not step up in execution cost (the majority), the ones that did step up, did so at either 128, 256 or 384.

Over 2,547 tests of 100 files each, the total number of occurrences of the execution cost stepping up was 0. Not a proof that it could never happen, but it seems reasonably dependable.

What if my app can create more files than this?

You’re likely interested in having a say in the performance of your file operations. Fortunately, this is easily done with a little help.

I’ll be posting a light weight helper class which allows you to declaratively control how many files are created per directory and hide this detail from the calling code. Here are some comparative results to demonstrate what is possible.

This benchmark was designed to be long running to find if any undiscovered limitations existed. Besides hitting an interesting issue with Visual Studio output copy/pasting for the logged results, this ran unhindered until the device was out of storage space.

And 380,000 files later… still humming along happily with negligible change in performance characteristics. Interestingly whilst there is a slight change in the consistency of the CreateFile() results, the OpenFile() results were unaffected by this.

All this with adding just 3 lines of declarative code to the app.

Are there any other limits to be aware of?

If you were inclined to ignore the performance degradation described here and allow directories to grow without limit, then it seems an exception will occur eventually: IsolatedStorageException – Operation not permitted on IsolatedStorageFileStream

In all benchmarks allowing files to be created without limit to the number of files per directory and without running out of storage space on device, an exception was raised after a little more than 10,000 files. In the case of 1k and 5k sized files the exception occurred after 10837 files were created and in the case of 50k sized files the exception occurred after 10826 files were created.

Conclusion

There are still opportunities for further optimisation, if you want ultra performant code for handling file create/open. The greatest gains have been accomplished here and using abstraction can be so done with minimal effort.

All results shown are the result of benchmarks executed on a Samsung Taylor 650Mhz Windows Phone 7 prototype device, OS Version 7.0.7004.0.

@MickNDev

Blogging WP7 Development Experiences

It’s been for some time that I’ve wanted to get this blog running to share experiences I have developing with WP7 that are relevant to the wider developer community.

What I hope you’ll find if you subscribe to the blog are articles exploring unique development experiences and from time to time, commonly saught after information.

Between…

• familiarising with pretty much every issue raised by developers using the WP7 tools on a daily basis since March 2010 on

1. MSDN – Windows Phone 7 Dev Forums
2. App Hub – Windows Phone 7 Dev Forums
3. Stack Overflow – Windows Phone 7 Dev Q&A

• contributing to building the base of reference question and answers that for time to come, developers will be able to rely on when having problems

• MSDN WP7 Dev: ~600 answers (ranked 1 for answers from April 2010 till apphub migration).

• AppHub WP7 Dev: 8k+ (ranked 1 top movers through to early 2011)
• 

• Stack Overflow WP7 Dev: 400+ answers
• 

• monitoring the platform’s progress in the press

• reviewing much of the interesting WP7 developer content created in the blogosphere

• and forwarding via Twitter (@MickNDev) the articles which stand out as either

1. interesting / innovative developer articles
2. new information that gives developers a perspective on how the platform is progressing
3. information that’s commonly saught after
4. and sometimes just content that will help people to get started with the tools

to the broader #wp7dev and local #wp7au developer communities.

…it’s been something of a challenge finding the time to get the blog up among these and other commitments. Not the least demanding of which has been preparing my first article, to follow, from some quite interesting experiences I’ve had with benchmarking recently into a widely consumable format. Deciding on a blogging platform and making a first cut at achieving a few styling goals (hope I didn’t do too badly there!) was a bit of a roadbump too that is now thankfully done – at least good enough for now (welcome to send me suggestions to improve that!).

I’ll get a contact page going when time permits… in the meantime feel free to tweet, dm or email me via the addresses I leave in my msdn/stack overflow forum profiles linked above.

I look forward to your reflections on the experiences I write about.

Best Regards,

Mick Norman.

• Recent Posts

  • Thoughts on Marketplace App Spam
  • Inside the Mango Developer Tools (App Platform, Communications, Search) – 3 quick channel9 sessions
  • Mango to Support Access to Hidden WiFi networks
  • NoDo Caching Observations for Windows Phone 7 Isolated Storage File Handles and an Update on Performance for Folders with Lots of Files (build 7390)
  • Congrats to the Windows Phone Team (listening to devs)
  • How many files are too many files for Windows Phone 7 Isolated Storage folders (performance)?
  • Blogging WP7 Development Experiences

• 
  Join the conversation